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FOREWORD 


This  technical  report  was  prepared  by  Dr.  Clifford  D.  Fawcett 
of  the  Deputy  for  Development  Planning,  Aeronautical  Systems  Division, 
Wrlght-Patterscn  Air  Force  Base,  Ohio,  ana  was  presented  to  the  Depart¬ 
ment  of  Industrial  Engineering  of  the  Ohio  State  University  In  partial 
fulfillment  of  the  requirements  for  the  degree  of  Doctor  of  Philosophy. 

This  work  was  directly  motivated  by  experience  obtained  while 
employed  as  a  member  of  the  technical  staff  at  Wrlght-Patterson  Air 
Forca  Basa.  In  performing  effectiveness  analyses  of  various  Air  Force 
weapon  systems.  It  has  become  Increasingly  apparent  that  the  tactics 
used  in  employing  the  system  can  be  of  overriding  Importance  In  de¬ 
termining  system  effectiveness.  It  is  also  clear  that  uncertainty 
is  an  unavoidable  and  crucial  factor  In  decisions  relating  to  future 
military  systems.  It  Is  relatively  easy  to  point  out  and  discuss 
these  facts  and  few  rational  people  will  dispute  their  Importance, 
but  it  seems  to  be  rare  for  a  system  evaluation  to  Include  explicit 
consideration  of  tactics  optimisation  and  uncertainty.  This  Is  un¬ 
doubtedly  due  to  the  conceptual  and  mathematical  difficulties  that  are 
encountered  In  so  doing,  coupled  with  the  practical  considerations  of 
limited  resources  and  time  available  for  most  weapon  system  evalua¬ 
tions.  With  this  situation  in  mind,  the  work  that  is  de6cribad  here¬ 
in  was  ladder taken  as  a  more  extensive  consideration  of  these  problems 
than  la  usually  possible. 
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Publication  of  this  technical  documentary  report  does  not  con< 
stltuta  Air  Force  approval  of  the  report's  findings  or  conclusions. 


It  la  published  only  for  the  exchange  and  stimulation  of  ideas. 
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ABSTRACT 


This  work  applies  dynamic  programing  and  teat  notions  from 
decision  theory.  Basic  recursive  relations  are  developed  for  determine 
istic  and  Markovian  decision  processes.  Sufficient  conditions  are 
stated  that  assure  the  optimality  of  results  that  these  relationships 
produce.  The  application  deals  with  the  p rob lea  of  asking  a  rational 
selection  of  tactics  for  air-to-ground  attack  when  faced  by  uncertainty 
as  to  the  exact  conditions  that  prevail. 

A  single  alroraft  attack  on  a  target  is  referred  to  as  a  "duel." 
A  duel  is  treated  as  a  Multistage  decision  process  with  successive 
aircraft  pastas  at  the  target  corresponding  to  stages  In  the  decision 
process.  The  basic  factors  to  be  considered  at  each  stage  of  the  duel 
are  the  weapon  effectiveness  as  a  function  of  the  number  of  weapons 
delivered,  the  aircraft's  ability  to  survive,  and  the  aircraft's 
ability  to  acquire  the  target  and  deliver  weapons.  We  seek  to  deter¬ 
mine  an  optimal  policy  that  indicates  the  number  of  weapons  to  be 
delivered  and  the  mode  of  attack  to  be  used  at  each  pass  depending  on 
what  state  of  affairs  develops  as  the  duel  progresses. 
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The  selected  policy  maximises  the  aircraft's  return  subject  to 
constraints  on  the  number  of  passes  that  can  be  made,  the  number  of 
weapons  available  to  the  aircraft,  and  the  probability  of  the  aircraft 
surviving  the  duel.  Several  different  types  of  return  to  the  attacker 
are  considered.  These  Include  the  expected  value  of  the  number  of  hlt< 
achieved,  the  probability  of  at  least  one  hit,  and  the  expected  utility 
of  the  duel  to  the  attacker  where  the  utility  Is  an  arbitrary  function 
of  the  number  of  hits  achieved. 

A  principle  result  from  the  duel  models  is  an  Indication  of  the 
maximum  return  as  a  function  of  the  constraining  probability  of  the 
aircraft  not  surviving  the  duel.  This  Is  referred  to  as  a  "retum- 
versus-attrltlon  function"  for  the  duel.  Multiple  aircraft  raid  models 
are  developed  to  determine  which  point  on  the  retum-versus-attrltton 
function  Is  the  best  operating  point  for  attacking  the  target.  These 
raid  models  assume  that  the  aircraft  In  the  raid  nuke  stochastically 
Independent,  statistically  Identical  attacks.  By  using  the  return- 
versus-attrition  function  from  a  single  aircro.lt  duel  model,  and  con¬ 
sidering  the  probabilistic  survival  of  area  defenses,  the  optimum  raid 
size  and  the  best  policy  for  the  duel  are  determined.  This  determina¬ 
tion  minimises  the  expected  value  of  the  number  of  aircraft  lost  in 
achieving  a  required  level  of  return  to  the  attackers. 

A  multiple  aircraft  raid  on  multiple  targets  Is  considered  by 
starting  from  the  previously  stated  basic  assumption.  Here,  the  problem 
Is  to  allocate  e  given  man  be r  of  aircraft  among  targets  and  specify  the 
policy  for  each  duel  to  maximize  the  total  utility  to  the  attackers 
subject  to  constraints  on  the  number  of  aircraft  available,  the  expected 
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value  of  the  number  of  aircraft  lost,  the  number  of  passes  an  aircraft 
can  make  against  each  target,  and  the  number  of  weapons  an  aircraft  can 
carry  to  each  target. 

The  question  of  what  tactic  to  choose  In  the  face  of  uncertainty 
as  to  the  true  parameter  values  la  approached  by  associaelng  a  range 
of  uncertainty  with  each  of  the  input  parameters.  We  assume  complete 
Ignorance  of  the  value  that  the  parameters  might  take  within  their 
respective  ranges  of  uncertainty.  A  systematic  method  is  developed 
that  aids  the  decision  maker  in  choosing  a  nominal  set  of  input  values 
such  that  the  solution  that  is  optimum  for  those  nominal  input  values 
constitutes  a  rational  tactic  selection  considering  that  the  reallted 
or  actual  input  values  might  fall  anywhere  within  their  ranges  of 
uncertainty. 
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dn  -  A  component  of  Dn;  number  of  weapons  allocated  to  pass  n. 

dn*  -  Best  salvo  size  at  pass  n. 

^  s  -  Increment  of  variation  for  s. 

Efl(m)  -  Expected  losses  per  raid  when  the  m1-*1  attack  policy  is 
used . 

E(^t  -  Constraint  on  the  expected  losses  in  attacking 

targets  i,---,t, 

-  An  arbitrary  limiting  value  associated  with  the  set  Sg. 

F(a;a')  ”  A  measure  of  system  performance  versus  a  when  using  the 
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with  a* . 
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h(4  ;dR)  -  Probability  function  of  number  of  hits  for  a  salvo  of 
size  dn. 

I  -  Maximum  value  of  i. 

I  -  Markov  state  index;  i  -  1 , , I . 

J  -  The  number  of  components  of  W. 

Ko(m)  -  Probability  of  achieving  at  least  one  hit  per  duel  when 

the  mth  attack  policy  is  used. 

KDt^mt^  “  Probability  of  at  least  one  hit  per  duel  with  target  t 
under  attack  policy  mt. 

Kj{  -  Required  probability  of  getting  at  least  one  hit  per  raid. 

Kg(a;a*)  -  Probability  of  at  least  one  hit  per  raid  versus  a  when 

using  the  tactic  that  is  based  on  the  input  values 
associated  with  a' . 


1 

1 


KRt(mc)  . 


LR(a;a’)  - 
% 

■t 

n 

Plj(Dn)  - 
nn(i;0)  - 

^nl^xn»*n^ 


R 

R* 

R(m) 


n 


rH<V 

rK(dn}  ■ 
rn  *®n ^ 


Probability  of  at  least  one  hit  per  raid  on  target  t 
under  attack  policy  rat. 

A  component  of  Dn;  mode  of  attack  index  at  pass  n. 

Best  mode  of  attack  at  pass  n. 

Expected  losses  per  raid  when  the  optimum  attack 
policy  (m*)  is  used. 

Expected  losses  per  raid  versus  a  when  using  the  tactic 
that  Is  based  on  the  input  values  associated  with  a*. 

Expected  losses  in  killing  a  target  with  repeated  raids 
when  the  optimum  attack  policy  is  used. 

The  relative  importance  of  target  t. 

Optimal  attack  policy  for  target  t. 

Stage  index;  n  »  1 ,N. 

Markov  state  transition  probability. 

The  probability  function  of  i  with  parameter  6. 

-  The  actual  probability  of  surviving  passes  n , - , 1 

versus  i,  xn,  and  sn. 

a  parameter  of  the  example  salvo  effectiveness  function 
(see  equation  (III-16)). 

Number  of  aircraft  per  raid. 

Best  raid  size  (  -  R(m*)). 

Raid  size  required  to  realize  CR  or  KR  as  appropriate 
when  the  mth  attack  policy  is  used. 

Raid  size  for  target  t. 

Optimal  raid  size  for  target  t. 

Constraint  on  the  total  aircraft  allocated  to 
attacking  targets  l,---,t. 

Expected  hits  per  salvo  of  size  dn. 

Probability  of  at  least  one  hit  in  a  salv"  of  size  dn. 

-  Stage  n  return  versus  X,,  and  D,,. 
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SA  -  Probability  of  the  aircraft  surviving  the  area  defenses 

one  way,  either  from  base  to  target  or  from  target  tc  t>ase. 

SAt  -  Probability  of  the  aircraft  surviving  area  defenses 

enroute  to  or  returning  from  target  t. 

Sjj(m)  -  The  actual  probability  of  the  aircraft  surviving  the 
duel  when  the  mth  attack  policy  is  used. 

SD  (X_)  -  A  set  depending  on  Xn  where  Dn€  Sq  (Xn). 

n  n 

SQt(mt)  -  Actual  probability  of  the  aircraft  surviving  a  duel  with 
target  t  under  attack  policy  mt. 

c  — 

—  -  A  set  whore  £  £  s-  . 

ERt  Rt  ERt 


"  A  Sfet  where  n,t^Smt(RRfRt>- 


SR  (Rt)  -  A  set  where  Rt€ 


S-  -  A  set  where  R„£  S-  . 

Rt-  t  Rt 


-  Probability  of  aircraft  survival  to  the  point  of  weapon 
release  on  a  pass. 

Su  -■  Conditional  probability  that  the  aircraft  survives  a  pass 

given  that  it  survives  to  the  point  of  weapon  release. 


-  An  arbitrary  set  where  Xp£  S^, 

Ss  (Xp)  -  A  set  depending  on  Xp  where  5niS3n(xn), 


-  An  arbitrary  limiting  value  associated  with  Ss  . 


sn  -  Constraining  probabilitv  of  the  aircraft  surviving 

passes  n,-«- ,1. 

t  -  Index  on  target  when  multiple  cargets  are  considered. 

t  «  1*“““*T. 

CpOfp.Dp)  -  State  vector  transformation. 

0  -A  parameter  of  the  example  salvo  effectiveness  function 

(see  equation  (III-18)). 
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INTRODUCTION 
The^J Problem 


Central  3tat:«!oen<: 

This  work  denis  irith  the  prob?«j.  <>£  staking  a  rational  selection 
of  tactics  for  air-to-ground  Attack  whetj  faead  by  uncertainty  ae  to  the 
exact  conditions  that  prevail-  The  intention  is  to  provide  a  syste¬ 
matic  method  for  making  the  best  tise  of  available  information.  The 
result  Jb  not  an  "automatic  tactics  selection"  but  rather  it  is  a 
quantitative  theoretical  structure  that  can  serve  as  a  fr&scvork  within 
which  to  evaluate  the  multitude  cf  tangible  wtd  intangible  factors  that 
must  be  considered  in  planning  an  air  strike. 

Teralnology  ar«d  Central  Concept 

To  begin  the  discussion  we  will  establish  same  terminology  and 
a  general  concept.  A  "raid"  will  denote  a  multlrle  aircraft  attack 
against  a  target.  The  raid  Is  cos  prosed  of  "sorties,"  where  each  sortie 
involves  one  aircraft  which  takes  off,  proceeds  to  the  target,  tafceo 
part  in  the  attack  on  the  target,  returns  to  its  base,  and  lands. 

When  an  individual  aircraft  reaches  the  target  area,  its  encounter  with 
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the  target  end  target  defenses  will  be  referred  to  as  a  "duel."*  A 
duel  may  Involve  multiple  passes  by  the  aircraft.  A  duel  might  be  fur¬ 
ther  described  as  follows. 

A  military  aircraft  with  a  given  number  of  bombs  on  board  is  to 
attack  a  defended  target.  A  maximum  of  N  passes  can  be  made  subject  to 
fuel  limitation  or  arbitrary  policy.  A  given  pass  may  include  acquir¬ 
ing  the  targot,  surviving  to  the  point  of  weapon  release,  releasing 
veaponsj  and'  surviving  the  pullout.  Some  Important  considerations  are 
aa  follows. 

When  the  aircraft  makes  a  pass,  target  acquisition  is  thought  of 
as  ar.  event  tlict  has  occurred  when  the  pilot  has  sufficient  Information 
to  allow  weapons  to  be  delivered.  This  could  Imply  that  he  visually 
sees  the  target  or  a  designated  ala  point.  It  could  also  Imply  that 
some  sensor  such  as  a  radar  has  produced  a  desired  response.  Once  tar¬ 
get  acquisition  has  occurred,  the  aircraft  Is  maneuvered  Into  position 
ar;d  aligned  for  weapon  delivery.  Because  of  constraints  on  the  aircraft 
maneuver  capabilities  and  limitations  on  pilot  reaction  time,  it  is 
possible  for  target  acquisition  to  occur  too  late  to  allow  for  a  weapon 
delivery  on  the  same  pass. 


*The  word  "duel"  when  used  In  this  work  has  a  slightly  modified 
meaning  from  the  more  conventional  use  of  the  word.  In  a  classical 
duel,  as  described ,  for  example,  by  Williams  and  Ancker  (26)  the  two 
duelists  fire  at  each  other  until  one  is  killed.  In  the  duels 
described  herein,  an  aircraft  attacks  a  ground  target  while  the  target 
defenses  fire  at  the  aircraft.  All  of  our  duels  end  if  the  aircraft  Is 
killed.  Some  of  our  duels  end  If  the  aircraft  achieves  a  specified 
objective  and  others  of  our  duels  proceed  independent  of  the  success 
of  the  aircraft's  attack  on  the  target. 
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We  will  refer  to  the  following  two  types  of  passes.  An 
"Initial"  pass  is  nude  when  target  acquisition  has  not  occurred  during 
any  previous  pass  of  the  saae  duel.  A  "subsequent"  pass  Is  made  when 
target  acquisition  has  occurred  on  at  least  one  previous  pass  of  the 
same  duel.  We  distinguish  between  an  Initial  pass  and  a  subsequent 
pass  because  target  acquisition  may  be  less  difficult  If  It  has 
occurred  on  a  previous  pass  of  the  saae  duel. 

The  probability  that  acquisition  occurs  on  an  Initial  pass  will 
be  symbolised  P(AC)  and  the  probability  that  target  acquisition  occurs 
on  a  subsequent  pass  will  be  symbolised  P(Aj).  The  symbols  P(A0D)  and 

P(Ajd)  will  denote  the  probability  that  acquisition  and  delivery  occurs 

o 

on  an  Initial  and  subsequent  pass,  respectively. 

The  notion  of  aircraft  survival  and  Its  relation  to  the  capa¬ 
bility  of  the  aircraft  to  attack  the  target  is  cosiplicated  by  the 
variability  In  possible  damage  to  the  aircraft.  "Damage”  may  result 
in  anything  from  Immediate  disintegration  of  the  aircraft  to  a  slight 
degradation  of  performance  or  even  to  no  effect  on  the  aircraft  capa- 

i 

btllty.  We  will  make  an  abstraction  of  the  survival  aspect  of  the 
problem  by  defining  the  probability  Sj  as  the  probability  of  surviving 
to  the  point  of  weapon  release.  We  assume  if  the  aircraft  survives, 
its  performance  is  completely  unaffected  and  if  It  does  not  survive, 
then  the  aircraft  does  not  participate  further  in  Che  attack  and  will  ' 

be  considered  as  a  loss.  Also,  we  define  Su  as  the  conditional  proba¬ 
bility  of  the  aivcraft  surviving  the  pass  given  that  It  survives  to 
the  point  of  weapon  release. 
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If  B  denotes  an  event,  the  notation  P(B)  will  denote  the 
probability  of  occurrence  of  the  event  B. 

I 
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The  third  principal  aspect  of  a  pass  concerns  weapon  effective¬ 
ness.  The  group  of  boaibs  delivered  on  a  given  pass  will  be  referred 
to  as  a  "salvo."  We  can  visualise  the  Individual  Impact  points  of 
boabs  In  a  salvo  as  clustering  aroistd  the  effective  aim  point  for  that 
salvo.3  The  location  of  the  effective  aim  point  varies  with  respect  to 
the  target  according  to  sons  probability  distribution.  The  function 
r(d)  will  represent  the  effectiveness  of  a  salvo  as  a  function  of  salvo 
slse,  d.  It  is  not  the  purpose  of  this  work  to  derive  the  function 
r(d),  but  It  is  Important  to  note  chat  r(d)  is  in  general  a  monotoni- 
cally  nondecreasing,  concave  function  of  d.  In  other  words,  larger 
salvos  have  greater  effectiveness  but  there  Is  a  diminishing  marginal 
return  as  salvo  slse  increases. 

The  term  tactics  when  used  in  this  report  Includes  the  following. 
The  operational  planner  must  specify  the  number  of  aircraft  per  raid 
or  "raid  slse."  A  policy  oust  be  established  as  to  the  maximum  number 
of  passes  per  duel  and  the  allocation  of  weapon  load  among  those  passes. 
A  policy  must  also  be  established  to  tell  the  pilot  how  to  make  each 
pass,  l.e.,  high  level,  low  level,  dive,  etc.  All  these  Items  must  be 
specified  In  a  rational  way  In  the  face  of  uncertainty  as  to  the  exact 
conditions  that  prevail. 


The  variation  of  Individual  Impact  points  from  the  aim  point 
siay  be  unintentional  as  would  be  caused  by  factors  such  as  ballistic  dis¬ 
persion.  The  variation  of  Individual  impact  points  from  the  aim  point 
may  also  be  Intentional  as  would  be  caused  by  Introduction  of  a  syste¬ 
matic  delay  In  the  time  of  release  for  the  various  weapons.  This  latter 
Is  generally  referred  to  as  "atlck  bombing,"  however,  we  will  refer  to 
any  group  of  bombs  released  on  a  pass  as  a  salvo. 


i 

i 


t 


A  rational  selection  of  tactics  must  involve  a  criterion  or 
objective.  The  specific  objective  can  vary  from  one  problem  to  another. 
The  basic  approach  adopted  In  this  report  is  to  plan  the  raid  so  as  to 
minimize  the  losses  sustained  In  achieving  a  given  level  of  effective¬ 
ness.  A  number  of  measures  of  effectiveness  are  considered.  For  some 
targets,  the  level  of  damage  is  proportional  to  the  timber  of  hits 
achieved.  This  might  be  true  of  a  large  area  target.  Another  canon 
class  of  targets  consists  of  those  such  as  a  revetted  artillery  site 
for  which  a  direct  hit  will  deactivate  the  site  and  any  miss  will  prob¬ 
ably  leave  the  site  unharmed.  For  these  targets,  the  level  of  effec¬ 
tiveness  is  in  terms  of  the  probability  of  at  least  one  hit.  Ue  can 
conceive  of  another  class  of  targets  which  In  Itself  seems  to  be  of 
largely  theoretical  lr  .erest  but  is  worth  including  because  it  makes  a 
convenient  introduction  to  the  most  general  case.  For  these  targets 
achieving  C  hits  is  adequate;  there  is  no  additional  value  in  achieving 
more  than  C  hits,  and  achieving  less  than  C  hits  is  of  no  value.  All 
of  the  preceding  measures  of  effectiveness  are  special  cases  of  the 
general  case  where  the  level  of  damage  depends  on  the  nvaaber  of  hin 
achieved  according  to  some  arbitrary  utility  function.  The  Implica¬ 
tions  of  these  various  measures  of  effectiveness  are  discussed  in  later 
chanters. 


Motivation 

User  Oriented 

The  motivation  for  this  work  comes  from  two  sources.  The  most 
obvious  Is  the  user  of  tactical  air-to-ground  weapons  systems.  As  a 
highly  technological  nation,  we  tend  to  invent  new  hardware  items  to 
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meet  military  problems.  It  seems  clear  that  research  should  also  be 
devoted  to  learning  hoe  to  more  effectively  use  existing  equipment. 
This  Is  one  of  the  goals  of  this  research. 
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Evaluating  System  Designs 

A  second  and  perhaps  more  motivation  for  this  work  comes 

from  the  need  to  perform  comparative  ^valuation  of  new  weapon  system 
designs.  This  is  the  aspect  from  which  the  weapon  system  designer 
views  the  problem.  In  evaluating  competing  new  designs  or  modifications 
to  existing  systems,  many  factors  muse  be  considered.  These  include 
cost,  operational  effectiveness,  maintenance  Implications,  logistics 
Implications,  training  implications,  and  delivery  schedule.  This  re- 
port  deals  with  operational  effectiveness. 

A  Two  Stage  Decision  Problem 

Cne  way  to  compare  the  operational  effectiveness  of  alternative 
system  designs  is  to  formulate  a  two  stage  decision  problem.  At  the 
first  stage  is  the  aggregation  of  decisions  that  determine  the  charac¬ 
teristics  of  the  weapon  system.  These  decisions  will  be  referred  tu 
as  design  decisions.  They  are  thought  of  as  being  made  by  a  largely 
fictitious  individual  to  be  referred  to  as  the  "designer."  At  the 
second  stage,  we  consider  the  operational  use  of  the  system  that  is  the 
product  of  the  first  stage.  The  second  stage  decisions  are  made  by 
the  "user." 

The  designer's  decision  problem  at  the  first  stage  can  usefully 
be  abstracted  In  the  terminology  of  Luce  and  Raiffa  (19)  as  an  Indi¬ 
vidual  decision  under  uncertainty.  The  states  of  nature,  0j,  where 
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j  is  an  integer  such  that  1  <  j  <  J,  are  identified  as  an  exhaustive 
and  mutually  exclusive  set  of  situations  that  represent  the  prospective 
usage  of  the  system.  Various  designs  aj  where  1  <  i  <  I  constitute  the 
alternatives  under  consideration.  A  utility  is  associated  with  the 
employment  of  design  alternative  1  in  situation  j. 

If  a  subjective  probability  distribution  pj  where  1  <  J  <  J  can 
be  defined  over  the  0 j ,  then  the  most  desirable  design  alternative  can 
be  chosen  by  maximising  the  expected  utility  or  by  some  other  means 
based  on  probabilities,  i.e.  ,  by  making  a  decision  under  risk.  If  the 
designer  is  completely  ignorant  of  the  probabilities  p j ,  or  chooses  to 
ignore  any  such  information  that  he  may  have,  then  the  design  decision 
is  made  as  a  decision  under  uncertainty.  Some  principle,  such  as 
maxim in  utility,  minlmax  regret,  the  principle  of  insufficient  reason 
or  the  pessimism-optimism  index,  might  be  applied.  Whether  or  not  the 
subjective  probability  distribution  can  be  defined,  a  key  element  of 
the  designer's  decision  is  the  set  of  utilities,  u^. 

To  determine  the  values  for  the  first  stage  decision,  the 
second  stage  decision,  i.e.,  the  user's  decision,  muse  be  considered. 
The  utility  of  a  given  system  design  in  a  particular  situation  depends 
on  the  design  of  the  system,  the  objectives  toward  which  use  of  the 
system  is  directed,  the  nature  of  the  situation,  and  the  maimer  in 
which  the  system  is  used.  The  user's  decision  problem  is  visualised 
as  a  constrained  optimization  in  which  tactics  are  selected  to  maxi¬ 
mize  the  utility  (ujj)  within  the  constraints  imposed  by  the  system 
design  (aj)  and  the  situation  As  an  example,  the  user  might 

wish  to  maximize  the  probability  of  kill.  The  type  of  aircraft  and 
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mode  of  usage  dictate  the  fuel  capacity  and  rate  of  fuel  consumption. 

The  situation  controls  the  base  to  target  distance.  These  factors 
combine  to  provide  a  constraint  on  the  nmnber  of  passes  that  can  be 
made.  Thus ,  the  user  must  maximise  the  probability  of  kill  subject  to 
a  constraint  on  the  number  of  passes. 

The  Implications  of  Tactics  Selection 

The  operational  effectiveness  of  a  weapon  system  in  a  particular 
situation  can  be  greatly  Influenced  by  the  choice  of  tactics.  It  Is 
therefore  Important  when  evaluating  a  weapon  system  to  use  the  tactics 
that  are  best  for  that  particular  design  and  situation.  For  example, 
consider  a  night  attack.  Suppose  two  design  alternatives  are  being 
compared  and  that  they  are  identical  with  the  following  exceptions. 
Suppose  design  A  provides  an  additional  special  sensor  that  has  a  highly 
accurate  target  locating  capability  but  has  a  short  range.  Suppose  the 
alternate  design  B  simply  provides  one  additional  bomb  and  relies  en¬ 
tirely  on  the  aircraft's  radar  for  target  acquisition.  Much  current 
practice  is  to  make  only  one  pass  at  the  tar  et  per  sortie.  If  the 
comparison  were  made  on  this  basis,  the  spei  f.al  sensor  of  design  A  may 
be  useless  because  its  short  range  means  tr[  ‘  the  information  it  pro¬ 
vides  cases  too  late  to  be  useful.  Thus,  since  design  B  provides  an 
extra  bomb  and  the  designs  are  the  same  otherwise,  design  B  will  look 
better.  If,  on  the  other  hand,  each  design  is  used  with  its  own  best 
tactics,  tiie  tactic  for  design  A  might  be  to  use  the  special  sensor  to 
locate  the  target  on  the  first  pass  and  than  deliver  weapons  on  the 
second  pass.  Design  B  might  still  call  for  just  one  pass.  On  this 
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basis,  design  A  may  or  nay  not  look  better  than  design  8  but  the  com¬ 
parison  certainly  stems  more  reasonable. 

The  principle  that  each  design  alternative  ahould  be  evaluated 
using  its  own  best  tactic  ia  not  new.  The  difficulty  is  date reining 
what  are  the  best  tactics.  Determining  the  best  tactic  for  a  given 
system  design  in  a  particular  situation  can  be  a  aajor  constrained 
optimisation  problem.  The  objective  function  and  the  constraints  oust 
be  carefully  formulated  and  a  solution  mist  be  found. 

When  evaluating  a  system  design,  it  is  sometimes  tempting  to  by- 
pass  the  tactics  optimisation  problem  by  having  a  "panel  of  experts" 
choose  the  tactics  appropriate  for  each  situation  in  which  the  system 
is  to  be  evaluated.  It  might  then  be  argued  that  If  all  designs  are 
evaluated  using  the  same  tactic  in  a  given  situation,  the  comparison 
is  "fair."  This  is  simply  not  true  os  wes  qualitatively  indicated  in 
the  example.  It  might  also  be  argued  that  the  panel  of  experts  can 
assign  each  system  its  own  best  tactics.  This  approach  has  a  number  of 
limitations.  The  designs  being  evaluated  are  generally  different  from 
existing  equipment  and  the  situations  of  interest  are  usually  beyond 
the  experience  of  any  panel  of  experts.  Further,  even  when  applicable 
experience  is  available,  existing  practice  is  not  necessarily  opt lam 1. 
Finally,  the  limitations  of  people  in  Judging  the  Implications  of 
complex  quantitative  relationships  are  wall  known. 

The  tactics  optimisation  problem  seems  to  be  inseparable  from 
the  design  evaluation  problem  whenever  the  user  has  some  latitude  of 
choice  as  to  the  manner  of  system  employment.  In  a  sense,  we  might 
view  the  role  of  the  designer  as  that  of  establishing  constraints 
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within  which  the  user  must  operate.  Thus,  selecting  a  best  system 
design  Is  in  effect  a  problem  of  deciding  which  set  of  alternative 
modes  of  use  should  be  provided  to  the  user.  The  cost  of  the  system 
generally  Increases  as  the  variety  of  modes  of  use  available  to  the 
user  is  Increased. 

The  Implications  of  User  Uncertainty 

Whether  we  are  considering  the  user's  problem  or  the  design 
evaluation  problem  which  contains  the  user's  problem,  a  considerable 
complication  is  Introduced  by  the  user's  uncertainty.  The  prospective 
usage  of  the  system  is  characterised  by  an  array  of  situations.  De¬ 
termining  the  utility  of  a  given  system  design  in  a  particular  situa¬ 
tion  can  be  created  as  a  conceptually  simple  constrained  optimisation 
problem  If  the  situation  is  exactly  defined.  Unfortunately,  the  situa¬ 
tion  is  not  exactly  defined  in  actual  practice.  Such  quantities  as 
the  probability  of  aircraft  survival  during  a  phase  of  the  sortie  are 
generally  matters  of  considerable  uncertainty.  Thus,  the  specification 
of  a  situation  must  generally  reflect  the  degree  of  uncertainty  in¬ 
volved  when  values  are  given  for  the  characterising  parameters. 

From  the  standpoint  of  the  user,  a  systematic  method  should  be 
available  for  considering  the  implications  of  uncertainty  and  his 
options  to  control  the  outcome  by  appropriste  tactics  selection.  From 
the  standpoint  of  the  designer,  it  is  important  to  understand  the  im¬ 
plications  of  user  uncertainty  because  it  affects  the  utility  of  a 
system  design  In  a  particular  situation.  A  general  principle  might  be 
that  there  is  no  point  in  providing  the  user  with  options  that  he  can't 
use  effectively  because  he  is  uncertain  of  the  situation. 
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Research  Philosophy 
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A  Concept  of  Operations  Research 

One  view  Is  that  operations  research  is  basically  a  conjee  ura l 
discipline  which  involves  investigating  the  consequences  of  assua  trlons. 
Accordingly,  three  aspects  of  this  discipline  night  be  identified 
First  is  the  specification  of  assumptions,  second  is  the  develop*  it 
and  employment  of  investigative  techniques,  and  third  is  the  appl  na¬ 
tion  of  theoretical  results  in  making  judgments  about  practical  r  1 
world  problems.  All  or  some  combination  of  these  aspects  may  be  .  » 
proprlate  to  a  given  study. 

Assumptions  are  invariably  involved  in  any  operations  resej  > 
Investigation.  These  assumptions  Should  be  such  that  knowledge  ot 
their  consequences  has  sane  value.  If  they  lead  to  a  theoretical 
structure  that  parallels  some  real  world  situation,  then  perhaps  I  - 
ferences  can  be  drawn  about  relationships  and  the  consequences  of  ts 
in  the  real  world  by  studying  the  corresponding  relationships  in  t 
theoretical  structure.  The  notion  is  that  studying  the  theoretica 
structure  Is  more  convenient  and  cheaper  than  studying  the  corresp  *i- 
ing  real  world  situation.  This  Is  particularly  true  of  military 
problems. 

The  investigative  techniques  used  in  a  study  can  range  from 
operational  experiments  to  mathematical  analysis.  We  might  chink  c 
an  operational  experiment  as  an  attempt  to  establish  a  relationship 
between  the  theoretical  structure  and  the  real  world,  i.e.,  hypothe  as 
or  results  deduced  therefrom  are  tested.  At  the  other  extreme,  tut  ’« 
statical  analysis  deals  entirely  with  logical  relationships.  Betweei 
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these  methodological  extremes  are  Investigative  techniques  such  as 
gaming  and  simulation. 

Using  theoretical  results  to  make  judgments  about  real  world 
problems  la  the  payoff  and  presumably  the  purpose  of  operations  research 
activity.  Whether  this  judgment  is  made  by  the  operations  researcher 
or  by  someone  else  is  not  a  central  issue.  The  important  point  is  that 
same  form  of  application  should  be  made.  In  this  regard  the  following 
quotation  from  Flagel,  Huggins,  and  Roy  (6)  is  pertinent:  "The  charac¬ 
teristic  of  a  true  operations  research  study  is  that  It  provide  realis¬ 
tic  answers  to  an  actual  practical  problem.  In  this  context  the  tools 
and  techniques  used  should  never  be  limiting;  the  goal  is  to  select 
techniques  that  allow  all  significant  factors  of  the  actual  problem  to 
be  considered. *» 

The  Nature  of  this  Work 

In  this  work,  assumptions  are  set  forth  that  lead  to  a  struc¬ 
ture  resembling  a  particular  real  world  military  situation.  Recursive 
analysis  methods  and  same  notions  from  decision  theory  &  e  used  to 
determine  and  express  the  consequences  of  these  assumptions.  The  pur¬ 
pose  is  to  explicitly  define  the  methods  that  are  used,  show  how  they 
apply  in  deducing  the  consequences  of  a  particular  set  of  assumptions, 
and  quantitatively  illustrate  the  nature  of  the  consequences  by  way  of 
numerical  examples. 

This  is  a  theoretical  study  that  points  out  how  a  given  type  of 
structure  can  be  usefully  analysed.  It  therefore  seems  important  that 
we  seak  generality  and  flexibility.  The  results  should  show  how  to 
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approach  a  variety  of  related  problems.  We  seek  problem  solving  tech* 
nlques  more  than  specific  analytical  or  nimerical  results. 

It  is  not  our  purpose  to  make  judgments  about  specific  real 
world  problems.  This  is  properly  the  province  of  those  who  are  directly 
involved  in  the  activities  that  are  being  studied.  This  might  be  the 
concern  of  on  operations  research  activity  that  is  an  integral  part  of 
the  military  organisation  that  is  conducting  the  activities  of  interest. 
A  certain  amount  of  foundation  can  be  laid  in  an  isolated  academic  an* 
vlronment  but  if  the  model  Is  to  finally  be  truly  effective,  it  and 
the  research  that  goes  along  with  it  must  be case  a  part  of  the  using 
organisation.  In  a  sense,  the  one  who  makes  the  application  must  also 
be  a  researcher.  He  must  modify  and  continually  develop  the  theory. 

Consider  the  notion  of  model  ’Validation"  in  the  light  of  the 
preceding  discussion.  The  model  would  be  accepted  as  valid  If  Its 
structure  parallels  the  real  world  situation  of  interest  sufficiently 
accurately  to  allow  useful  conclusions  to  be  drawn.  The  decision  maker 
must  decide  whether  or  not  this  is  true  of  a  given  model  in  a  given 
decision  situation.  Accordingly,  the  validity  of  an  operations  research 
model  is  meaningful  only  when  a  specific  decision  is  to  be  made;  other¬ 
wise,  there  is  no  basis  for  judging  whether  or  not  the  model  consti¬ 
tutes  a  satisfactory  representation  and  the  concept  of  validity  has  no 
meaning.  Since  we  are  not  making  decisions,  model  "validity"  will  not 
be  of  concern.  We  should,  however,  be  concerned  with  the  logical  con¬ 
sistency  of  the  theoretical  structure.  We  may  also  be  concerned  that 
real  world  situations  exist  for  which  setae  decision  maker  might  be 
willing  to  consider  our  structure  as  a  ’valid"  representation  whan 
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making  practical  decisions.  This  last  thought  emphasises  the  import* 
ance  of  generality  and  flexibility  In  our  methods. 

The  approach  that  is  developed  in  this  report  is  basically 
numerical.  Some  analytical  relationships  are  investigated  but  the 
complexity  of  the  problem  along  with  the  critical  need  for  flexibility 
of  the  model  seem  to  dictate  a  largely  r.merlcal  approach.  It  should 
be  noted,  however,  that  the  recursive  analysis  techniques  that  are  used 
are  subtile  and  powerful  computational  tools.  The  numerical  character 
of  these  techniques  seems  to  pose  no  great  practical  problem  since 
computers  are  widely  available  to  all  potential  users. 


Related  Literature 

The  air-to-ground  attack  problem  as  treated  in  this  work  does 
not  seem  to  be  a  very  popular  subject  in  the  open  literature.  There 
are,  however,  articles  which  relate  to  various  aspects  of  our  problem. 
The  most  applicable  of  these  will  be  cited  and  categorized  with  same 
Indication  of  how  they  relate  to  our  subject. 


Stochastic  Duels 


Uilliams  and  Ancker  (26)  have  developed  a  theory  of  "stochastic 
duels"  which  they  describe  as  follows: 

In  the  "fundamental"  duel,  two  duelists,  A  and  B,  fire  at  each 
other  uneil  one  is  killed.  A's  firing  time  (that  is,  the  time 
between  rounds)  is  a  random  variable  with  a  known  probability  den¬ 
sity  function,  fg(t).  Successive  firing  times  are  selected  from 
fA(t),  independently  and  at  random.  The  situation  is  the  same  for 
B  except  that  his  firing  time  has  a  different  density  function, 
fg(t).  Each  time  A  fires,  he  has  a  fixed  probability,  PA,  of 
killing  B.  Similarly,  B's  kill  probability  is  Pg.  After  the 
starting  signal,  each  contestant?  loads,  aims,  and  fires  the  first 
round.  That  Is,  in  the  "fundamental”  duel,  they  start  with 
unloaded  weapons.  Both  A  and  E  have  unlimited  supplies  of 
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ammunition  that,  among  other  things,  makes  a  kill  a  certainty. 

A  wins  if  he  is  the  one  to  score  a  kill.  The  probability  of 
this  will  be  called  F(A)  and  P(A)  *  P(B)  »  1. 

In  reference  26,  P(A)  is  determined  for  the  case  where  f  A(t) 
and  fg(t)  are  negative  exponential  distributions.  The  effect  of  giving 
one  contestant  a  random  initial  tine  advantage  is  also  Investigated. 

In  reference  3,  Ancker  extends  the  model  to  the  case  where  both  con¬ 
testants  have  limited  ansnunition  supplies.  He  determines  P(A),  P(B), 
and  the  probability  that  "they  both  run  out  of  ammunition.  Both  of 
these  papers  approach  the  problem  by  computing  the  distribution  of  time 
to  kill  for  the  two  contestants  and  then  determining  the  probability 
that  one  gets  a  kill  before  the  other  does. 

In  reference  4,  Ancker  and  Williams  consider  the  fundamental 
duel  with  discrete  firing  times  where  A  and  B  fire  at  fixed  but  possibly 
different  intervals  and  the  ammunition  supply  is  unlimited.  They  also 
consider  a  case  where  the  contestants  fire  simultaneously,  a  near  miss 
by  one  causes  the  other  to  lose  one  firing  turn,  and  cam unit ion  is 
unlimited.  Finally,  this  paper  considers  duels  that  are  not  one-on- 
one. 

Our  air-to-ground  attack  problem  might  be  considered  as  a  type 
of  duel  where  the  contestants  fire  at  each  other  simultaneously  each 
time  the  aircraft  makes  a  pass.  If  the  aircraft,  is  contestant  A  and 
the  target  with  its  defenses  is  contestant  B,  we  are  interested  in  P(A), 
the  probability  that  the  aircraft  kills  the  target  before  the  aircraft 
is  killed.  In  our  duel,  however,  contestant  A  has  a  limited  ammunition 
supply  which  he  can  fire  in  salvos  with  arbitrary  salvo  effectiveness 
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versus  salvo  slse.  We  are  Interested  In  maximising  P(A)  and  determin¬ 
ing  the  best  allocation  of  weapons  among  passes  on  the  part  of 
contestant  A.  This  type  of  duel  Is  developed  In  Chapter  V.  Our 
generalisation  of  the  above  type  of  duel,  which  Is  also  developed  In 
Chapter  V,  extends  the  aircraft's  options  to  Include  multiple  nodes  of 
weapon  delivery  and  It  considers  the  aircraft's  target  acquisition 
problem.  A  further  complication  of  our  duel  Is  that  simply  maximising 
the  aircraft's  probability  of  kill  without  regard  for  Its  probability 
of  survival  Is  not  appropriate.  Because  of  this,  we  maximise  the  air¬ 
craft’s  probability  of  kill  subject  to  a  constraint  on  its  probability 
of  survival. 

Some  further  papers  on  stochastic  duels  that  appear  In  the 
literature  Involve  the  distribution  of  the  nunber  of  rounds  fired, 
reference  1,  and  the  distribution  of  the  time  duration,  reference  2. 


Tactical  Air  Games 


Fulkerson  and  Johnson  (16)  formulate  the  following  tactical  air 
game  In  which  each  side  must  continually  allocate  available  aircraft 
between  counter  air  and  ground  support  missions.  They  describe  the 
formulation  as 

...  a  multi-move  game  In  which  both  sides,  at  each  period 
of  the  campaign,  simultaneously  deploy  their  forces  between  the 
two  missions.  Each  force  suffers  a  fixed  rate  of  attrition  per 
period  due  to  accidents,  etc.,  and  In  addition  loses  planes  pro¬ 
portionally  to  the  slse  of  the  enemy's  attack  on  his  air  fields. 
Replacements  for  each  side  are  received  periodwlse,  and  these 
■ay  be  functions  of  time.  The  payoff  is  assumed  to  be  the 
difference  between  the  total  number  of  ground  support  sorties 
flown  by  the  two  sides  during  the  campaign,  discounted  for  future 
time  periods. 

The  symetrle  case  In  which  the  attrition  rates  are  the  same  for  both 
sides  Is  solved  for  both  finite  and  infinite  campaigns. 
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Bellman  and  Dreyfus  (7)  show  how  to  treat  the  same  problem  by 
using  dynamic  programing.  The  model  is  further  discussed  and  developed 
by  Weiss  (25)  and  by  Berkovlts  and  Dresher  (8,9). 

Two  essential  inputs  for  this  model  are  the  red  and  blue  kill 
potential  per  plane  sent  against  the  op >onent’s  air  fields.  For  the 
symmetric  case,  these  are  equal.  Our  model,  particularly  the  duel  of 
Chapters  III  and  IV  (called  herein  the  E^  duel)  should  be  helpful  In 
evaluating  these  quantities.  Furthermore,  once  the  number  of  ground 
support  sorties  to  be  flown  has  been  determined  at  a  given  stage  of 
the  game,  it  is  necessary  to  allocate  those  sorties  among  the  proapec* 
tlve  targets  and  determine  the  best  tactics  for  each  raid.  This  problem 
is  treated  in  Chapter  VII  of  this  report. 

A 

Allocation  of  Weapons 

Marme  (20)  discusses  the  problem  of  allocating  a  number  of 
weapons  to  a  complex  of  targets.  He  reduces  this  problem  to  the  form 


n 

Minimize  £  a. 

J-l 

« •  p/J 

(1) 

n 

subject  to  £  y.  ■  m 

J-?  J 

(2) 

* 

IV 

o 

J  -  n 

(3) 

where  J  ■  l,»—,n  ■  the  index  on  targets 

aj  «  the  unit  worth  of  target  j 


She  notation  used  In  this  section  is  in  keeping  with  the 
weapon  allocation  literature  and  Is  generally  different  from  thit  in 
the  rest  of  the  paper  which  is  based  on  dynamic  programming  literature, 
l.e..  Ref.  17  and  Ref.  22. 


18 


Pj  »  the  probability  of  kill  for  any  weapon  versus 
target  J 

yj  -  the  number  of  weapons  assigned  to  target  J. 

He  then  shows  how  to  formulate  the  above  as  a  transportation  problem 
which  results  In  an  Integer  solution. 

G.  G.  den  Breeder,  Ellison,  and  Qnerling  (10)  also  consider  the 
weapon  allocation  problem  and  prove  some  applicable  theorems.  Using 
the  above  notation  where  It  Is  applicable,  they  first  assume  that 
pj  •  p2  -  ---  pn  •  p  and  consider  the  form 


n 


max  V  -  £  1 

k-1 

/k  ak  (yj, — ,yn> 

(4) 

subject  to 

n 

£  y,  -  m 

j-1 

(5) 

yj  i  0 

J  -  I , - ,n 

(6) 

where 

V  «  expected 

value  of  the  targets  destroyed 

Vk  -  the  value  of  destroying  exactly  k  targets 

ak(ylf - ,yn)  •  the  probability  of  destroying  exactly 

k  targets  as  a  function  of  the  allocation 

»yn). 

For  this  problem,  they  prove  the  following  theorems  1  and  II. 

"Theorem  I.  If  the  Vk  are  nondecreasing  functions  of  k,  then 
the  maximum  V  Is  attained  when  the  yj's  differ  by  at  most  one." 

"Theorem  II.  The  probability,  Pk,  of  destroying  k  or  more 
targets  la,  for  each  k,  a  maximum  when  the  y,'s  differ  by  at 
moat  one."  1 


* 
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They  also  consider  the  problem  defined  by  equations  (1),  (2)  and  (3). 
For  that  problem,  they  prove  that  the  following  theorem  III  holds 
for  all  Pj. 

"Theorem  III.  Given  that  ^  yj \  minimises 


a(m)  •  Z  a.  (1  -  p.)VJ 
j-1  3  3 

(7) 

subject  to  y j  >  0 

(8) 

n 

and  Z  yi  »  o 

J-1  J 

(9) 

then  ^  y^  minimizes 

a(mel)  •  Z  a,  (1  *  P<)^ 
j-1  3  1 

(10) 

subject  to  y'j  >  0 

(11) 

n 

and  Z  'y'.  -  m  ♦  1 

J-1 

(12) 

9j  •  yj  for  j  i*  k  and  yk  -  yk  ♦  1,  where  k  satisfies 

ak  Pk  “  i  <af<  n  {  -J  a-Pj)7j  Pj}  "  (13) 

w 

According  to  the  authors,  if  one  interprets  a^  (1  -  p^)  as  a  revised 
estimate  of  the  value  of  the  kth  target  based  upon  an  optimum  assign* 
ment  of  m  weapons,  the  procedure  implicit  In  theorem  III  snrely  states 
that  an  added  weapon  should  be  assigned  to  that  target  for  which  the 
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expectation  of  (revised)  value  destroyed  is  largest.  Thus,  by  starting 
with  a  ■  0,  weapons  can  be  added  one  at  a  time  in  such  a  way  that  the 
allocation  at  each  stage  is  optimal. 

Leans  and  David  (18)  extend  the  solution  to  the  case  where  there 
is  more  than  one  type  of  weapon  available.  Again,  using  the  foregoing 
notation  where  it  is  applicable,  their  problem  takes  the  form 


n  m 

Maximize  £  a.  [l  -  n  (1  - 

j-1  J  l-l 

»,/‘3  ] 

(14) 

n 

subject  to  £  y..  -  m. 

j-1 

i  -  1,— ,m 

(15) 

u 

IV 

© 

1 

»-* 

w 

1 

1 

1 

«• 

s 

(16) 

j  -  1, - ,n 

where  Pjj  -  probability  of  kill  for  weapon  type  1 
versus  target  j 

yjj  -  number  of  weapons  of  type  i  allocated 
to  target  j 

-  number  of  type  1  weapons  that  are  available. 

Their  approach  la  to  determine  values  L^  which  represent  the  number  of 
type  1  weapons  required  to  be  the  equivalent  of  one  type  i  weapon.  By 
using  this  device,  the  problem  reduces  to  the  form  of  equations  (1), 
(2)  and  (3)  that  was  considered  by  the  previous  two  references.  Lemns 
and  David  Indicate  the  possibility  of  solving  the  problem  by  the  two 
previous  methods  which  both  produce  integer  results.  They  also  offer 
a  solution  that  they  indicate  was  obtained  by  the  method  of  Lagrange 
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multipliers.  This  method  treats  the  weapons  being  allocated,  l.e., 
the  equivalent  number  of  type  1  weapons,  as  a  continuously  variable 
quantity.  The  answer  may  Involve  fractional  mssbers  of  weapons. 

Because  of  round-off  error,  this  approach  would  be  moat  appllcabl  If 
Che  number  of  weapons  being  allocated  was  large  compared  to  the  number 
of  targets. 

The  weapon  allocation  papers  that  have  been  discussed  can  be 
related  to  the  air  to  ground  attack  problem  as  follows.  Suppose  we 
wish  to  maximise  the  expected  value  of  the  number  of  hlta.  We  might 
do  this  If  the  attacker's  utility  la  a  linear  function  of  tha  number  of 
hits.  Thus,  we  have  the  problem: 

Maximise  E  •  £  SJ  r.(y.)  (17) 

J-» 

n 

subject  to  2  y.  •  ■  (18) 

j-1  ‘ 

Yj  >  0  j  -  1,— - ,n  (19) 

where  J  •  1 ,n  •  index  on  air  to  ground  pass 
S  -  probability  of  the  aircraft 
surviving  a  pass 

yj  -  the  timber  of  weapons  delivered 
on  the  jth  pass 

rj(yj)  ••  the  expected  value  of  the  number 
of  hits  on  pass  J  as  a  function  of 
the  number  of  weapons  delivered  on 


pass  j. 
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a  -  the  total  n unbar  of  weapons 
carried  by  tha  aircraft. 

Aaauning  stochastically  independent  delivery  errors  for  the  weapons 
that  are  delivered  an  pass  J,  we  have  the  special  case  where 

rj(yj)  “  1  -  (1  -  Pj)y-J  (20) 


and  equation  (17)  becomes 

n 

Maximise  2  ft 

J-i 


(l  -  Pj)yj  ] 


(21) 


which  has  the  same  solution  as 

Minimise  £  SJ  (l  -  P,)7j  (22) 

J-l  1 

which  the  same  as  (1)  if  Sj  Is  interpreted  as  S-*. 

The  problem  stated  in  equations  (17),  (18)  and  (lb)  is  essen¬ 
tially  the  same  aj  the  simple  EH  duel  of  Chapter  III.  The  recursive 
analysis  technique  that  Is  used  In  this  work  offers  a  practical  way 
of  obtaining  the  solution  with  no  special  restrictions  on  the  form  of 
the  function  'j(yj)  although  a  fora  similar  to  that  given  by  equation 
(20)  is  used  for  the  numerical  exaaples.  In  addition,  the  use  of 
recurs ive  analysis  makes  it  practical  to  consider  probabilistic  target 
acquisition,  multiple  modes  of  attack,  and  a  constraint  on  the  proba¬ 
bility  of  the  aircraft  surviving  the  duel.  Finally,  the  recursive 
techniques  allow  solution  of  the  problems  that  are  discussed  in 
Chapters  V  and  VI  which  are  not  treated  in  the  foregoing  articles. 
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Sequential  Dee  Is  ten  Processes 

The  basic  purpose  of  this  investigation  is  to  show  how  to  us  - 
fully  analyte  problems  related  to  the  selection  of  tactics  for  air-  on¬ 
ground  attack.  Dynamic  programming  is  an  extremely  useful  technlqu 
for  obtaining  solutions  to  these  problems.  The  dynamic  programming 
theory  that  is  discussed  and  used  herein  was  adapted  or  developed  o  ly 
as  needed  to  solve  the  specific  problems  at  hand.  These  problems  li 
volve  sequential  decision  processes  with  a  relatively  small  number  <  .? 
stages. 

The  "principle  of  optimality"  as  introduced  by  Bellman  (5)  it 
the  starting  point  for  developing  all  of  the  recursive  relationships 
that  are  discussed  herein.  The  technique  for  applying  the  principle  of 
optimality  to  solve  deterministic  sequential  decision  problems  is  di  •> 
cussed  by  many  authors  including  Bellman  (5)  and  Nonhauser  (22), 

Chapter  II.  Howard  (17)  extends  the  application  of  this  principle  t 
the  solution  of  sequential  decision  problems  involving  Harkov  procesi  os. 
He  provides  for  selecting,  at  each  stage,  the  best  act  from  an  array 
of  alternative  actions.  Nemhauser  (22),  Chapter  V,  discusses  a  still 
more  general  form  of  multistage  decision  model  which  applies  to  what 
he  calls  a  "stochastic  optimization  problem"  or  a  "multistage  optimi¬ 
zation  taider  risk."  He  introduces  a  random  variable,  1^,  at  stage  n 
whose  value  determines  the  stage  return  and  the  state  variable  trans¬ 
formation.5  He  formulates  the  case  where  the  kn  are  stochastically 

5To  illustrate  the  meaning  of  these  terms,  the  "stages"  gener¬ 
ally  correspond  to  passes  in  our  problems;  the  "stage  return"  is  the 
utility  derived  from  the  weepons  delivered  an  the  corresponding  pass; 
the  "state  variable"  is  a  vector  that  characterizes  the  state  affaire 
when  the  aircraft  is  preparing  to  make  a  pass. 
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Independent  from  stage  to  stage.  He  also  points  out  that  if  the  dia- 
tribution  function  of  kn  depends  on  knel*  the  value  at  the  previous 
stage,  the  process  is  Markovian.  The  probability  distribution  of  kn 
■ay  also  depend  on  the  values  taken  by  k  et  all  previous  stages  in 
which  case  the  process  is  not  Markovian. 

The  'toonotonlcity  assunption"  is  basic  to  establishing  the 
optimality  of  the  dynamic  programing  results.  This  assumption  was 
Introduced  by  Mitten  (21)  and  is  further  discussed  by  Denardo  and 
Mitten  (12)  and  by  Nemhauser  (22).  The  exact  statement  of  the  mono- 
tonlclty  assumption  varies  somewhat  with  each  author  making  the  state* 
ment  that  best  serves  his  own  purposes. 

Champs  and  Schroeder  (11)  discuss  sequential  decision  processes 
from  the  standpoint  of  multistage  games  or  stochastic  games.  Their 
paper  relies  heavily  on  the  work  of  Shapely  (23),  A  stochastic  game 
conslstc  of  a  series  of  stages  where  the  states  occupied  by  two  com¬ 
peting  players  are  subject  to  probabilistic  transition  from  stage  to 
stage  according  to  transition  probabilities  controlled  Jointly  by  the 
two  players.  Associated  with  each  transition  is  a  payoff  from  player 
two  to  player  one,  l.e.,  the  gome  is  zero  sum.  A  terminating  stochas¬ 
tic  game  is  one  in  which  at  each  stage  there  is  a  non-zero  probability 
of  the  play  ending.  Chames  and  Schroeder,  following  Shapely’s  develop¬ 
ment,  show  how  such  a  game  can  be  solved  by  Iterative  application  of 
linear  programming.  They  also  give  a  stopping  criterion  for  the 
Iterative  process. 

Of  particular  Interest  here  is  the  demonstration  by  Chames  a>>d 
Schroeder  that  when  player  one  knows  player  two’s  strategy  at  each 
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stage,  the  stochastic  game  reduces  to  the  discounted  Markovian  deci¬ 
sion  process  that  is  discussed  by  Howard  (17).  If  the  process  is 
truncated  so  there  is  a  finite  nixiber  of  stages,  Howard's  value  itera¬ 
tion  applies.  If  the  process  is  not  truncated  so  there  is  an  infinite 
nimber  of  stages,  Howard's  "policy  iteration"  algorltha  applies. 
Furthermore,  Chimes  and  Schroeder  show  how  linear  programing  can  be 
used  to  detertalne  the  optimn  stationary  policy  for  the  infinite  stage 
Markovian  decision  process. 

The  technique  for  applying  the  principle  of  optimality  to  our 
problems  is  discussed  in  Chapter  II.  The  basic  philosophy  for  dealing 
with  multistage  decision  problems  is  similar  to  that  of  Neahauser, 
however,  the  subject  Is  developed  in  such  a  way  as  to  considerably 
almpllfy  the  development  o2  appropriate  recursive  relationships  and 
hopefully  make  them  highly  intuitive,  fhls  simplification  is  Important 
when  applying  dynamic  programing  to  complex  problems.  The  monotonlclty 
assumption  is  stated  here  in  such  a  way  as  to  minimise  the  difficulties 
encountered  In  deciding  whether  or  not  it  applies  in  a  given  situation. 

The  treatment  of  Markovian  decision  processes  is  developed  in 
Chapter  II  to  the  extent  of  separating  the  state  vector  into  two  parts, 
one  of  which  Is  subject  to  probabilistic  transformation  while  the  other 
is  subject  to  a  deterministic  transformation  that  may  depend  on  the 
outcome  of  the  probabilistic  transformation.  This  innovation  results 
in  Improved  computational  efficiency  and  a  reduction  in  computer 
memory  requirements.  It  also  considerably  simplifies  the  mechanics 
of  applying  the  resulting  recursive  relationship  to  the  problem.  This 
technique  does  not  appear  in  the  lltoreture  that  has  been  reviewed. 
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There  also  appears  In  Chapter  II  of  this  work  an  "equivalence" 
assumption.  This  assumption  accompanies  the  aonotonicity  assumption 
in  establishing  the  optimality  of  the  dynamic  programing  results  ob¬ 
tained  for  Markovian  decision  problems.  This  assumption  is  not  made 
explicit  in  the  literature  that  has  been  reviewed,  although  it  is  in¬ 
variably  satisfied  by  the  recursive  relationships  that  appear.  A 
probable  reason  why  Che  equivalence  assumption  does  not  appear  in  the 
literature  is  that  most  discussion  of  Markovian  decision  processes  Is 
in  terms  of  additive  present  and  future  returns  (see  references  22,  17 
and  11).  In  that  Case,  the  equivalence  assumption  is  satisfied.  In 
this  work,  we  treat  the  general  case  where  present  and  future  returns 
are  not  necessarily  additive  (see  Chapter  V)  so  the  equivalence  assump¬ 
tion  becomes  Important. 


CHAPTER  II 

GENERAL  RECURSIVE  RELATIONSHIPS 

Analysis  of  Multistage  Systems 

The  purpose  of  this  chapter  is  to  develop  and  discuss  some 
generalized  recursive  relationships  that  will  be  used  In  later  chapters. 
This  discussion  will  introduce  some  terminology  and  notation.  It  will 
also  make  explicit  the  requirements  that  must  be  met  if  these  recursive 
relationships  are  to  serve  our  purposes. 

We  are  interested  in  making  decisions  relating  to  the  perfor¬ 
mance  of  serial  multistage  systems.  Nemhauser  (22)  defines  such  a 
system  as  "a  set  of  stages  Joined  together  in  series  so  that  the  out¬ 
put  of  one  stage  becomes  the  input  to  the  next."  Our  problems  Involve 
a  finite,  relatively  small  number  of  stages,  i.e.,  an  infinite  stage 
approximation  as  Introduced  by  Bellman  (5,  p.  11)  is  not  generally 
applicable. 

Consider  a  system  of  stages  indexed  n  •  1,  2,—  ,  N  where  each 
stage  in  the  system  is  characterized  as  follows.  The  state  of  the 
system  at  stage  n  is  completely  described  by  the  state  vector  Xj^. 

The  decision  made  at  that  stage  is  designated  by  the  decision  vector 
Dn.  The  return  realized  depends  on  the  state  and  the  decision  and  is 
denoted  by  the  function  rn(Xn,  Dn).  The  output  of  stage  n,  which  be¬ 
comes  the  input  p-ate  for  stage  n-1,  depends  on  the  state  and  decision 
at  stage  n  and  is  indicated  by  the  following  "transformation"  relation¬ 
ship. 
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Vl  -  fcn  <*n.  Dn> 

We  will  assume  that  t  (••)  Is  single  valued.  A  system  composed  of  a 
series  of  N  stages  might  be  illustrated  by  Figure  1  which  is  similar 
to  the  corresponding  diagrams  used  by  Nemhauser  (22). 

The  basic  analysis  approach  to  be  followed  Involves  composing 
or  "putting  together"  the  system  by  starting  at  stage  one  and  adding 
one  stage  at  a  time  to  the  already  existing  structure.  The  stages  will 
be  numbered  according  to  the  order  in  which  they  are  added  in  the  cam* 
position  process.  The  central  idea  of  the  composition  process  can  be 
described  as  follows. 

Suppose  we  have  an  existing  structure  of  n- 1  stages.  Since  the 
state  of  the  system  at  stage  n-1  is  completely  described  by  the  state 
vector  then  for  a  given  system  structure,  no  other  input  informs* 

tlon  is  required  to  determine  the  maximum  return  that  is  realizable 
from  the  existing  n-1  stages.  Let  us  designate  this  maxi mm  return 
from  the  n-1  stages  as  fn_j  (Xn_ j). 

We  now  wish  to  expand  the  existing  n-1  stage  structure  to  Include 
stage  n.  The  situation  might  be  illustrated  by  the  diagram  in  Figure  2. 
The  function 

Lr  v*n»^n^’  fn-l^*n-l^  “  ®n  t-^n’^n’  ^n-l^^n-1^ 

“  «n  CVE>n.  Vl  <S<VDn»3  (2) 

characterizes  the  composition  of  the  stage  n  return  with  the  maximum 
return  available  from  the  remaining  n-1  stages.  This  will  be  referred 


to  as  the  "return  function." 


*  / 
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Fig.  1.— Diagram  of  Multistage  Process. 
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Fig.  2. --The  Composition  Process. 
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The  maximum  composite  n  stage  return  Is  now  determined,  subject 
to  appropriate  conditions,  by  choosing  the  value  of  Dn  that  maximises 
the  n  stage  return  function. 

Cn«n>  “  «"  ®n  CW  fn-l<*n-l>] 

Dn 

Bellman  and  Dreyfus  (6)  indicate  that  the  basic  functional  equation  of 
dynamic  programing  has  the  form  of  equation  (3). 


A  Deterministic  Decision  Process 

A  deterministic  decision  process  will  be  defined  as  one  in  which 

the  transformation  relationship  of  equation  (1)  is  such  that  X  ,  is 

n—  l 

deterministically  known  If  and  Dn  are  known.  We  are  interested  in 
specifying  and  justifying  sufficient  conditions  such  that  if  they  are 
met,  recursive  application  of  equation  (3)  to  a  deterministic  decision 
process  will  yield  the  maximum  n  stage  return  and  the  optimal  decision 
rule  at  each  stage. 

Suppose  transformation  relationships  of  the  form  of  equation  (1) 

are  given.  Let  Sv  and  (X„)  be  seta  of  all  allowable  values  of  X 
*n  ^n  n  " 

and  Dn,  respectively,  at  stage  n  where  Sq  (X^  depends  on  XR.  Note 
that  these  sets  must  have  the  property  that  If  X^Sx  and  Dn£  SD  Wn)» 
then  tntXn.Djj)  €  ^  for  all  2  <  n  <  N. 

Define  the  one  stage  return  as  g^Xj.D^),  a  function  that  is 
defined  for  all  Xj£  and  Dj£  Sq^(Xj).  Then  the  function 


fitt,)  -  Max  g. (X. ,D  ) 


(<*> 


i«  defined  for  all  Xj£  ;  It  is  the  maximum  one  stage  return. 
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Now  let  fn-i(Xn.i)  h®  the  maximum  n- 1  stage  return  as  a  func¬ 
tion  of  Xn_ ^  and  let  it  be  a  function  that  is  defined  for  all 

i’  ^  —  n  —  N*  N°te  that  fn_i(Xn.i)  can  *>«  visualised  as  the 
result  of  having  maximised  same  objective  function  relating  to  the  n- 1 
stage  structure  subject  to  the  condition  that  the  state  of  the  system 
at  stage  n-1  is  completely  described  by  the  state  vector  X,,.}. 

The  return  function  gn  Lxn»Dntfn-i<Xn_j)]  characterises  the 
composition  of  the  return  from  stage  n  with  the  maximum  return  from  the 
remaining  n-1  stages.  Suppose  the  function  gn  ["‘3  has  the  following 
property: 

Monotonicity:  If  is  replaced  by  an  Independent 

variable,  say  y,  ?3  io  deflne<1  *nd  it  Is  * 

monotonleally  nondecreasing  function  of  y  for  all  3^6 
Dn€  sDn(xn>*  *nd  2  <  n  <  N. 

Then  It  follows  from  the  definitions  of  monotonicity  and  that 

the  function  represents  the  maximum  return 

that  is  obtainable  from  the  n  stage  system  for  given  X,,  and  Dn.  * 
Accordingly,  the  basic  functional  equation 

f  (X  )  -  Max  gn  [xn,Dn,fn.l<tn<Xn,Dn))]  (5) 

On€S„n(Xn) 

represents  the  maximum  n  stage  return  subject  only  to  the  value  of  Xn; 
the  function  fn(X„)  is  defined  for  all  X„€  Sj^,  2  <  n  <  N. 

*A  proof  of  this  statement  appears  in  Appendix  B. 


Thus  fj(Xj)  Is  the  max  lain  one  stage  return  and  we  have  shown 
that  under  suitable  conditions,  if  is  Che  maximum  n-1  stage 

return,  then  ^0^)  is  the  maximia  n  i^tage  return,  So  by  mathematical 
induction,  ^0^)  is  the  maximum  n  stage  return  for  all  X,^  and 
1  <  n  <  N. 

We  will  now  summarise  the  sufficient  conditions  for  the  opti¬ 
mality  of  fn(X_)  in  a  deterministic  decision  process.  For  completeness. 


*1  Cx:,Dj,f0(X0)]  =  gj  (Xj.Dj)  (6) 

for  all  X^S^  and  D1€SDi«1). 

If  the  following  transformation  relationships  are  given, 

*n-l  '  '-n  <VDn>  5  2  <  n  <  N 

►Aere  must  exist  sets  Sj^  and  Sq^CX,.)  such  that  if  X„€  and  Dn£  Sfl  (Xf) 
then  ^  for  all  2  <  n  <  N.  Also,  the  f  met  ion 

«1  [*l.°l«<o«o>l  must  be  defined  for  all  Xj^C  and  DjES^tXj). 
Furthermore,  the  function  gj^  [xn,Dn,tR(Xn_1)]  must  possess  the  mono- 
tonlcity  property  that  was  defined  previously  in  this  section  for  all 
X^'s  ^Xjj*  ®n^  ^n^»  and  2  <  n  <  N.  If  the  foregoing  conditions  are 

met,  then  recursive  application  of  equation  <5)  will  yield  the  maximum 
n  stage  return  for  all  1  <  n  <  N. 

A  Generalised  Markovian  Decision  Process 
In  the  deterministic  decision  process,  the  transformation  re¬ 
lationship,  equation  (2),  relates  the  s*:ate  of  the  system  at  stage  n-1 


MM  ^  If  »y  !■  ■wfwrr^’M'|l|»»y 


r 


deterministically  to  the  state  of  the  system,  Xn,  and  the  decision,  Dn, 
at  stage  n.  We  will  now  consider  the  situation  where  the  state  of  the 
system  at  stage  n- 1  is  a  random  variable  with  a  probability  function 
that  depends  on  TL^  and  Dn.  This  conditional  probability  function  will 
be  denoted  PCX^^  |  Xn;Qn) .  This  is  a  Markov  process  since 


pWn-l  1  xn;Dn^  "  p**n-l  |  xN,xN-l*"“"»xn;DN’DN-l 


1  <  n  <  N 


With  this  process  In  mind,  suppose  that  we  have  an  existing  n-1 
stage  structure.  Since  the  process  of  interest  is  now  a  stochastic 
process,  the  n-1  stage  return  for  a  given  value  of  is  a  random 

variable.  Let  denote  ehe  maximise  expected  value  of  the  n-l 

stage  return.  We  wish  to  expand  the  structure  to  Include  stage  n.  In 
general,  the  composite  return  of  stage  n  and  the  remaining  n-1  stages 
depends  on  X,^  Dn,  and  Xn_j.  Denote  this  composite  return  by  the  re¬ 
turn  function  gn  LXn»Dn*Xn-l’^n-l^Xn-1^3-  Then  the  maximum  expected 
value  of  the  n  stage  return,  subject  tc  suitable  conditions,  is  given 


f  (X)"d  £?%*_>  X-  Zs  P(Xn-llW 

Dn€sDn<xn>  ^-1^  SXn_1 


«n  CVBn-*n-l»f,e.l<*n.l>3  <8> 


For  our  purposes,  equation  (8)  is  the  basic  functional  equation  that 
applies  to  Markovian  decision  processes. 
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A  Special  Markovian  Peels  Ion  Process 

tie  will  now  discuss  a  special  case  of  the  Markovian  decision 
process  that  ts  ot  particular  interest  in  the  applications  that  are  to 
be  considered. 

Suppose  the  components  of  the  state  vector  can  be  divided  into 
two  sets  as  follows.  Suppose  the  first  set  consists  of  one  component, 
the  value  of  which  can  be  designated  by  the  Integer  variable  1, 

1  <  i  <  I.  The  variable  1  will  be  said  to  designate  the  Markov  state 
of  the  system  (for  reasons  that  will  became  clear).  The  remaining  com¬ 
ponents  of  the  state  vector  completely  describe  the  state  of  the  system 
in  all  respects  other  than  the  Markov  state.  The  vector  composed  of 
these  remaining  components  will  be  referred  to  as  the  "residual  state 
vector"  and  will  be  designated  1^. 

Now  suppose  that  transitions  among  the  Markov  states  are  proba¬ 
bilistic  and  are  governed  by  a  stochastic  matrix  of  transition  proba- 
olllties  having  elements  designated  Pjj(Dn),  i.e.,  Pij(Dn)  is  the 
conditional  probability  of  being  in  Markov  state  J  at  stage  n-1  given 
that  the  system  Is  in  Markov  state  i  at  stage  n  when  decision  Dn  is 
made.  Let  the  transformation  of  the  residual  state  of  the  system  be 
deterministic  for  given  n,  i,  j.  The  residual  state  of  the  system  at 
stage  n-1  is  determined  by  the  transformation  relation 

Vl’  -  tnij  K*  Dn>  w 

Thus,  we  have  probabilistic  transition  among  the  Markov  states 
which  controls  the  transformation  of  the  residual  state  vector. 
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The  foregoing  can  be  related  to  the  basic  functional  equatl  \ 

(8)  a3  follows.  Recalling  the  notation  that  was  described  in  the  ’rst 

paragraph  of  the  previous  section,  the  conditional  probability  fun  cion 

2 

that  appears  in  equation  (8)  can  be  expressed  as 


p<Vi  I  V°k«>  -  Pttn-i.J  I 

-  Pl«n-i  |  P2(J  I  C*>Dn> 

Now  considering  equation  (9), 


uO) 


pl<Xn-l  |  "  1'° 


11) 


if  Xn_J  is  the  value  given  by  equation  (9)  and  Pl(Xn_]:  J  j,X^,i;Dn'  is 
zero  otherwise.  Thus,  equation  (10)  can  be  written  as 


P<Vl  |  W  -  P2<J  I 


<  -■) 


Now  assuming  that  the  value  taken  by  j  is  stochastically  independer 
of  X^,  i.e. ,  the  Markov  state  at  stage  n-1  is  independent  of  the  re 
sldual  state  at  stage  n,  equation  (12)  be cooes 


P«n-1  |  VDn>  “  P2<J  |  ‘i°n>  S  PiJ<°n> 


t  0 


2The  quantity  p(Xn»i»j  i  X^,i;Dn)  is  a  conditional  Joint  pi* 
biiity  function  in  the  rand  obi  variables  and  J.  The  second 

equality  in  equation  (10)  follows  from  a  relation  in  basic  probahll  v 
theory.  If  u  and  v  are  random  variables  with  the  Joint  prohibit i  > 
function  h(u,v),  then  h(u,v)  ■»  hj(u  j  v)  h2  (v)  where  hj^u  |  v]  • 
the  conditional  probability  function  of  u  for  given  v  and  h2(v)  is  e 
marginal  probability  function  of  v. 


■■•iW-  WtlH-HMiHWMWt  l|JK1*IS:l.ltlUUUt(lM.  \Uiut 
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Here  we  have  Introduced  a  convention  chat  Markov  state  variables  and 
stage  indices  will  appear  as  subscripts  while  residual  state  variables 
and  decision  variables  will  appear  as  arguments.  This  convention  will 
be  followed  henceforth. 

Since  the  Markov  state  variables  are  to  appear  as  subscripts, 
the  factor  g^  that  appears  in  equation  (8)  becomes 

4  [••••]  "  «nlj  C<,Dn.Xn:i.fn.l,J<Xnll)3 

and  If  la  replaced  by  Its  equivalent  from  equation  (9), 

*11  “  Snlj  C*n»Dn*^n«l,j ^cnlJ 

Finally,  for  this  special  case,  equation  (8)  becomes 

cnl  <x’>  £  PlJ<°n)  *nij  l»n.*n-l,  J  <cni  j  (X>n»]|  (16) 

Note  that  the  prime  on  the  g^j  £«♦••]  In  equation  (14)  does  not 
appear  In  equations  (15)  and  (16)  because  the  gnij  L“"J  that  appears 
In  equations  (15)  and  (16)  represents  a  functional  fora  that  Is  differ¬ 
ent  from  the  functional  form  represented  by  the  gnJj  that  appears 

in  equation  (14), 

We  will  now  proceed  to  specify  and  Justify  sufficient  conditions 
such  that  if  they  are  net,  recursive  application  of  equation  (16)  will 
result  In  values  of  fnj(X^)  that  represent  the  maximum  expected  value 
of  the  n  stage  return. 

Suppose  transformation  relationships  of  the  fora  of  equation  (9) 
are  given.  Let  and  Sjj  <X^)  be  sets  of  all  allowable  values  of 
and  Dn  respectively  at  stage  n,  where  Sq  (X^)  depends  on  X„.  These 


sets  mast  have  the  property  that  If  X^£S^  and  Dnf  S0  (X^),  then 
tnijO^.OnJCSj^  t  for  all  1  <  1  <  I,  1  <  J  <  I,  2<n<N. 

Define  the  one  stage  return  as  (xJ,Dj) ,  a  function  that  Is 
defined  for  all  Xj^S^^DjC  Sjj  (x[),  1  <  1  <  I,  1  <  j  <  i.  Then  the 
function 

1 

fli<Xi)  "  n  42,  P1J<X*1>  *lij(Xi'Dl)  d7> 

is  the  maximun  expected  value  of  the  one  stage  return;  it  is  defined 
for  all  Xiesx,,  1  <  1  <  X. 

Now  let  fft_j  .(Xn’j)  be  the  maximum  expected  value  of  the  n-1 
stage  return  and  let  it  be  a  function  that  is  defined  for  all 

,  1  <  J  <  I,  2  <  n  <  H.  Note  that  j  tXj^)  can  be 

visualized  as  the  result  of  having  maximized  some  objective  function 
relating  to  the  n-1  stage  structure  subject  to  the  condition  that  the 
Markov  state  variable  takes  the  value  j  and  the  residual  state  vector 
takes  the  value  X^_j  at  stage  n-1. 

The  return  function  g^j  [x^,Dn,fn-lj jtX^ characterizes  the 
composition  of  the  return  from  stage  n  with  the  maximum  expected  value 
of  the  return  from  the  remaining  n-1  stages.  Suppose  that  for  all 
^£Sx;'DnfSDn^'  1  <  i  <  I,  1  <  i  £1.  2<n<N,  the  return 
function  L***0  l,as  t*,e  f°H°wing  two  properties; 

1.  monotonlclty;  If  f n. j ^ j l )  1®  replaced  by  an  Independent 
variable,  say  y,  gnlj  Cxn»Dn»x3  is  defined  and  it  is  a  mono- 
ton  leal  ly  nondecreasing  function  of  y. 
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2.  equivalence:  If  Z  is  a  random  variable  and  E(Z)  repre¬ 

sents  the  expected  value  of  z,  then 

E  t8nlJ0C>n,Z)]  -  [»n.E(Z)] 

Given  these  properties,  the  function  gnJj  repre¬ 

sents  the  maximum  expected  value  of  return  that  is  obtainable  from  the 
n  stage  system  for  given  values  of  X^,Dn,l,j.^  It  follows  that  the 
functional  equation 


fMj«  ) 


Max 

OnCSojXn)  * 


pij(Dn*  gnij  CXn»Dn»fn-l,J^tniJ^Xn*Dn^3 


represents  the  maximum  expected  value  of  the  n  stage  return  subject  to 
the  condition  that  the  residual  state  vector  takes  the  value  X^  anri  the 
Markov  state  variable  takes  Che  value  1  at  stage  n.  The  function 
fr>,(Xn)  1*  defined  for  all  X^Cs^,  i  <  i  <  I,  2<n<N. 

• 

Thus ,  fi|(X})  is  the  maximum  expected  value  of  the  one  stage 

return  and  we  have  shown  that  if  fn.ifjOCn^1)  i3  the  maximum  expected 

value  of  the  n-l  stage  return,  then  fftj(X^)  is  the  maximum  expected 

value  of  the  n  stage  return  subject  to  suitable  conditions.  So  by 

mathematical  induction,  fnl(X^)  is  the  maximum  expected  value  of  the 

n  stage  return  for  all  X^C  S^’ »  1  <  i  £  I*  l<n<N. 

n  ” 

The  sufficient  conditions  for  the  optimality  of  will  now 

be  summarised  for  the  special  Markovian  decision  process.  For  camplete- 


ness,  let  ^  [x>x,fOJ«;>]  =  gpjCXpD^ 
for  all  X^,Dpi,J  in  their  respective  sets. 


3  A  proof  of  this  statement  appears  in  Appendix  B. 
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If  we  are  given  the  transformation  relationships 

Cl-  WCV  *  2<n<N  (20) 

1  <  i  <  1 
1  <  j  <  I 

there  must  exist  sets  of  allowable  values  of  X^  and  Dn,  and 

SD  (X^),  such  that  if  X^€SX,  and  Dn6  Sp  (X^),  then  t^.,  (X;,l>n)  e  S*, 
n  r\  n  n- 1 

for  all  1  <  1  <  I,  1  <  j  <  I.  Also,  the  function  gltj  £xj,Di,f0(X0>3 
must  be  defined  for  all  xjfiS^.Dj^SQ  (xj),  1  <  1  <  I  and  1  <  J  <  I. 
Furthermore,  the  function  C^n*Dn*^n-l  j^^n-1^  muSt  possess  the 

monotonlclty  and  equivalence  properties  that  were  defined  previously 
in  this  section  for  all  X^£  S^,  Dn£  Sp  (X^),  1  <  1  <  I,  1  <  J  <  I, 

2  <  n  <  N. 

If  the  foregoing  conditions  are  met,  then  recursive  application 

of  equation  (18)  will  yield  the  maximin  expected  value  of  the  n  stage 

return  for  all  X^£  Sj, ,  1  <  1  <  I,  l<n<N. 
n 


Markov  Decision  Process  with  Unobservable 


toy  State  Transitions 


The  discussion  so  far  has  presupposed  a  knowledge  of  the  state 
of  the  system  at  all  stages.  It  will  be  useful  for  later  reference  to 
Indicate  a  functional  equation  for  a  case  where  the  Markov  state 
transitions  are  not  observable,  hence  the  Markov  state  of  the  aystem  is 
only  probabilistically  known  for  n  <  N. 

Let  TT^d)  be  the  probability  function  of  the  Markov  state  at 
stage  N.  Let  nn(l;  ttn,  D^,  —  ,Dn+l^  b®  ***•  probability  function 


jt.  [Jth'icfilllrler  <('. 
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of  i  at  stage  n.  The  function  nn(i;  ttn,Dn,  DN.i»“-“.Dn+i)  car  be 
evaluated  by  multiplication  of  n^U)  and  the  succeeding  matrices  of 
transition  probabilities. 

The  functional  equation  for  this  situation  can  be  developed  by 
starting  from  equation  (18).  Since  the  Markov  state,  1,  is  only  known 
probabilistically  at  each  stage,  the  maxinum  n  stage  return  will  be 
taken  as  the  maximum  expected  value  of  the  return  taken  over  the  random 
variable  1.  This  maximum  n  stage  return  depends  on  the  Initial  proba¬ 
bility  function  tt^  and  it  depends  on  the  decisions  made  at  stages  N, 
N-l,—  ,n»l;  it  will  be  denoted  fn(X^,  nN,  DN,— Dn+l).  The  return 
function  may  depend  an  the  stage  and  the  Markov  states  involved;  hence, 
the  return  function  is  deflnod  as  g^j  [^X^»Dn,fn_1(Xn2l*nN,DN,—  ,Dn>3- 
The  quantity  i  •>  in  equation  (18)  thus  becomes 


pIjtDn)  ^ij  LXn*Dn*fn-l*Xn-l,1TN»DN» 


Since  this  quantity  is  the  maxlmm  expected  value  of  the  n  stage  return 
for  given  n,  xj,,Dn,  and  i  and  since  i  Is  a  random  variable,  the  maxi¬ 
mum  expected  value  of  the  n  stage  return  for  given  n,  X^,  and  Dn  is 
given  by 

1 

2  TTn  (i;  ttn,  dn,  -iDj^j) 

i»l 

Taking  the  maximum  over  Dn  gives 

I 

fn(xn»  "n*  dN»  °N- 1 »Dn+l^  ”  Max  2  ^n(1;TTN*°N*DN-l»“"“Dnel^ 

°n^Dn(Xn)  ‘-1 

I 

^  piJ(DnJ  «nij  IX  *  Dn»  fn-l<*n-l*  nN»  °N»  °N-1» - *DnJ]  <2l> 


» 


t 


2 ^  wiw  •• 


The  high  dimensionality  of  equation  (21)  will  probably  render  it  lm< 
practical  to  implement  for  most  cases,  but  it  is  useful  to  have  an 
indication  of  che  type  of  relationship  that  is  involved. 


If  an  Implementation  of  equation  (21)  were  undertaken,  it  would 
involve  determining  nn(l;  tt,  for  all  feasible  sequences 

of  Dn  and  for  all  n.  The  results  could  then  be  used  in  equation  (21) 
to  determine  the  optimum  decision  at  stage  n,  Dn*(X^,  rr^,  D^,  D^p 
»DnelJ*  The  residual  state  variable  transformation  relations  could 

<£  tC 

then  be  used  to  determine  the  opcl mum  sequence  (D^,---!^  ). 


CHAPTER  III 


THE  SIMPLE  Eh  DUEL1 
Recursive  Relationships 

Having  developed  some  basic  functional  equations,  we  will  now 
consider  how  they  might  be  usefully  applied  to  the  air-to-ground 
attack  problem.  The  number  of  hits  achieved  by  the  aircraft  Is  a 
random  variable.  This  chapter  considers  situations  where  the  return 
of  Interest  Is  the  expected  value  of  the  number  of  hits.  The  cost  is 
In  terms  of  the  ntmber  of  aircraft  lost. 

The  first  problem  will  be  a  very  simple  duel  In  which  we  seek 
to  maximise  the  expected  value  of  the  number  of  hits  (hereafter  re¬ 
ferred  to  as  "expected  hits")  achievable  by  a  single  aircraft  attacking 
a  defended  ground  target.  This  will  be  referred  to  as  the  "simple  EH 
duel." 

Suppose  an  aircraft  with  xN  bombs  on  board  Is  to  make  not  more 
than  N  weapon  delivery  passes,  a  salvo  of  dfl  bcn’-'t  it  t«  be  delivered 
on  Che  pass  n  subject  to  the  restrictions  that 
N 

2  «*n  <  *N  «> 

n»l 

dn  >  0  1  <  n  <  N  (2) 

1 Through out  this  work,  a  duel  Involving  allocation  of  weapons 
among  passes  and  not  Including  probabilistic  target  acquisition  or 
multiple  modes  of  attack  Is  referred  to  as  a  "simple”  duel. 
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Throughout  this  work,  and  dn  are  assisted  to  be  discrete 
variables.  The  unit  of  weapons  will  normally  be  referred  to  as  a  bomb; 
however,  it  could  represent  a  cluster  of  boobs  or  a  salvo  of  rockets. 

Let  ST  represent  the  probability  that  on  a  given  pass,  the  air¬ 
craft  survives  long  enough  to  deliver  weapons.  Let  Su  represent  the 
conditions,  probability  that  the  aircraft  survives  the  pass  given  that 
it  survives  long  enough  to  deliver  weapons  on  that  pass.  In  this  con¬ 
text,  we  will  interpret  "not  surviving"  as  the  occurrence  of  an  eneoy 
action  that  prevents  the  aircraft  from  participating  further  In  the 
attack.  This  could  siean  anything  from  Immediate  kill  of  the  aircraft 
to  relatively  minor  damage. 

To  apply  recursive  analysis  to  this  problem,  a  pass  will  be 
identified  as  one  stage.  It  Is  not  necessary,  but  It  will  be  convenient 
to  number  the  passes  In  reverse  order  so  that  the  chronologically  last 
pass  Is  pass  number  one.  The  state  of  the  system  when  preparing  for 
pass  n,  l.e.,  at  stage  n,  can  be  completely  defined  by  specifying  the 
number  of  weapons  remaining,  denoted  xn.  Thus  will  be  the  ttate 
variable  playing  the  role  that  the  state  vector,  Xjj,  played  In  the 
general  formulation  of  Chapter  II.  Let  the  number  of  weapons  delivered 
on  pass  n,  dn,  be  the  decision  variable  analogous  to  Dn. 

If  dn  weapons  are  delivered  on  pass  n,  then 

*n-l  "  xn  ”  dn  ;  1  <  n  <  N  (3) 

Thus,  since  the  state  variable  la  subject  to  deterministic  transforma¬ 
tion,  we  have  a  deterministic  decision  process.  In  studying  this  pro- 
,  the  stage  return,  rnWn),  will  be  the  expected  hits  per  selvo 


cess 
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as  a  function  of  salvo  sice.  Ths  function  fn(*n)  will  represent  the 
max  1 aim  expected  hits  achievable  on  the  remaining  n  passes  If  there  are 
Xh  weapons  remaining. 

To  satisfy  the  sufficient  conditions  for  optimality,  we  must 

show  the  existence  of  sets  Sx  and  S<j  (xn)  such  that  xn C  ^  and 

n  n  An 

dn^-sdn  ^or  1  <  n  <  N.  It  can  be  seen  by  inspection  of  the  trans¬ 
formation  relationship,  equation  (1),  and  considering  the  non-negatlvlty 
of  dn,  that  the  sets 


and 


\  “  {xn:  *n€^0,l. - 


;  1  <  n  <  N  (4) 


i„(xn)  “  {^€{o.U - .*nj|  J 


1  <  n  <  N  (5) 


satisfy  the  requirement.  The  quantity  XN  >  0  is  the  largest  number  of 

2 

weapons  that  might  be  of  interest  at  pass  N. 

The  return  function  must  be  formulated  next.  If  one  pass  re¬ 
mains  and  dj  weapons  are  to  be  delivered,  then  the  expected  hits  la 
the  probability  of  surviving  to  the  point  of  weapon  delivery  times 
r||(di);  leftover  weapons  have  no  value.  Thus, 


gl  C*i»  dl»  Wl  "  ST  rR<dl>  J  n  -  1  (6) 


where  the  functions  In  equation  (6)  must  be  defined  for  all  Xj£Sx 


1 


and  dj£  S^Cxj). 


20nce  a  value  of  xN  Is  established,  the  actual  ntaber  of  weapons 
Available  et  stage  N,  xN,  can  take  any  value  such  that  <  x^.  The 
value  of  xN  only  establishes  the  range  of  values  of  xN  for  which  solu¬ 
tions  will  be  obtained. 


US 


When  sore  Chen  one  pear  remains,  the  composition  of  the  maximum 
n-1  stage  return,  w,th  1,1  •  n  return,  rjj(dn),  can  be 

accomplished  as  follows.  The  expected  hits  for  pass  n  la  the  proba¬ 
bility  of  surviving  to  the  point  of  weapon  release  times  the  expected 
hits  obtained  on  the  salvo  that  is  delivered.  The  maximum  exp-Cted 
hits  fot  the  remaining  n- 1  passes  Is  the  probability  of  surviving  pass 
r  times  fn_i(^n-i).  Thus, 

*n  Cxn*  V  Cn-l<xn-l^  "  ST  rH(dn>  ♦  ST  Su  fn-l<xn-l>  *  ?  <  n  <  » 

(7) 

By  examining  equation  (6),  we  see  that  if  r^(dj)  la  defined  for 
all  XjC Sx^  and  dj£Sd^(xj),  then  the  return  function  for  n  ■  1  Is  de¬ 
fined  for  all  XjCSx  and 

Furthermore,  when  2  <  n  <  N,  we  see  from  equation  (7)  tf.ai  If 

rH(dn)  is  defined  for  all  x^Sg  and  dn€  Sd  (xn) ,  then  the  return 

n  n 

fimction  meets  the  monotonicity  requirement  that  Is  defined  In  Chapter 
II  for  a  deterministic  decision  process.  The  previous  statement  Is 
true  because  S-j.  >  0  and  su  ?.  «• 

Note  that  the  function  gn  £*'*J  Is  Indexed  by  stage  so  Its  form 
and  Its  associated  coefficients  can  therefore  be  made  stage  dependent. 
Accordingly,  the  function  rH  and  the  quantities  ST  and  Su  could  carry 
the  subscript  n.  This  will  be  understood  to  be  true  throughout  this 
work  but  the  subscript  Is  not  carried  explicitly  because  It  would  stake 
the  notation  more  awkward  without  adding  significantly  to  the  content 


of  the  work. 
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We  have  now  defined  sets  and  return  functions  so  that  the  suf¬ 
ficient  conditions  for  optimality  are  satisfied.  Recursive  appltca- 
tlon  of  equation  (11-5)  will  yield  the  desired  fn(xn).^  Substituting 
equi  cions  (5)  and  (6)  Into  (II-*>  and  considering  equation  (4)  gives 

f^fxj)  ■  Max  r||(<!|)J  »  0  d  d  x^  (2) 

0  <  d!  d  ':j 

Since  rH(d)  Is  assmed  to  be  a  nondecreasing  function  of  d, 

fj(xj)  -  ST  rH(xj)  ;  0  <  <  5?n  (9) 

Substitution  of  equations  (3),  (5),  and  (7)  into  ( IT  —  5)  and  considering 
equation  (4)  gives 

fn(xn)  -  Max  [ST  rrt(dn)  ♦  ST  Su  fn.x(xn  -  dn)]  ;  0  <  x„  <  xN 
0  <  dn  <  *n 

2  <  n  <  N 

(10) 

Equations  (9)  and  (10)  constitute  the  recursive  relationships 
that  apply  to  the  simple  duel. 


Concavity  of  fn(xn) 

It  Is  generally  a  supportable  assumption  that  r^(dn)  Is  a  concave 
function  of  dn.  It  can  be  shown  that  when  this  is  true,  the  fn(xn)  de¬ 
termined  from  equations  (9)  and  (10)  are  concave  functions  of  xn  for  all 


The  equation  numbers  start  from  (1)  in  each  chapter.  When  re¬ 
ferring  to  an  equation  of  the  current  chapter,  its  arable  number  will 
be  used.  When  referring  to  an  equation  of  a  previous  chapter,  its 
arable  number  will  be  preceded  by  the  appropriate  Reman  numeral  chapter 
number. 


I  t 

i  l 


l  <  n  <  N.  This  can  be  proved  as  follow.  First,  if  n  -  1,  equation 
(9)  shows  that  since  ST  >  0  and  rH(dj)  is  concave  in  dj,  then  fjdj) 

Is  concave  in  Xj. 

Now  if  2  <  n  <  N,  consider  equation  (10),  Note  that  if  rH(dft) 
is  concave  and  if  f^iOc,,  -  dn)  is  a  concave  function  of  xR  -  dn,  then 
since  Sf  >  0  and  Su  >  0,  the  expression 

S'  <xn«  dn>  “  ST  CrH<dn>  ♦  su  *  drt>l  <ll> 

is  a  concave  function  of  xn  and  dn<  To  show  this  for  a  given  n  let  Uj 

and  U2  be  arbitrary  values  of  xn  and  let  vj  and  V2  be  arbitrary  values 
of  dn  where  uj,  U2  £  and  vj,  V2€S<jn(xn).  For  0  <  A  <  l, 

g’(Xn.dn)  *  ST  CrH(Xvje(l-X)  v2)  *  sufn»l^ul**1"^  u2-Xv1-(l-A.)v2)] 

-  ST  {  r^v^O-A)  v2)  ♦  Sufn_j  [^(uj-Vj)  ♦  (1-X.)(U2-V2)]  j- 

>  ST  XrH(v!)  ♦  (1-A)  rH(v2)  *  sJ>fn.i(uj  -vj)  ♦  (1-A.)fn»  j («2*v2 ^3  J 

-  ST^\[rH(vl)  ♦  Su  fn.1(u1-v,)]  ♦  (i-X)[rH(v2)  ♦  Sufn.1(u2-v2)] J- 

-  Ag'  (u1,vl)  ♦  (l-A. )  g'<u2,v2)  (12) 

A  proof  given  by  Be 11a an  (S,  p.  21)  can  now  be  used  to  show 
that  the  following  is  a  concave  function  of 

f  (x  )  -  Max  g'Cxn.d,,)  (13) 

0  <  dn<  *n 


This  proof  by  Bellnan  is  included  here  rather  than  simply 
referencing  It  because  it  is  vital  to  Include  some  such  proof  in  the 
demonstration  of  concavity.  The  reader  way  not  have  ready  access  to 
Ref.  5  and  furthermore,  some  small  notatlonal  adjustments  have  been 
made  in  adapting  the  proof  for  our  purpose. 
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For  a  given  n  and  using  Che  previous  definitions  of  u^,  Vj,  v2,  X, 


fn(xn)  “  fn(Xul  ♦  u2>  "  Mjx  g' (Xu1«(l-X)u2,dn) 

0  <  dn  <  Xuj*(l-X)u2 

(14) 


The  quantity  dfi  can  be  replaced  by  dn  ■  Xv2  ♦  (1-X)v2  where  Vj  and  v2 
range  independently  over  the  intervals  0  <  vj  <  uj,  0  <  v2  <  u2.  Now 
for  given  X,  we  seek  optimising  values  of  v^  and  v2. 


fn(Xuie(l-X)u2 )  -  Max  g,[Xu1e(l-X)u2»^vl*(1"^^v2]  05) 

0  <  vi  <  uj 

0  <  v2  <  °2 

Since  g’(xn,  dn)  is  concave  In  xn,  dR, 

g^Xujed-XJuj,  Xvle(l-X)v2]  >  Xg'<U|(V|)  ♦  (1-X)  g'(u2,v2)  (16) 

Hence, 

fn(Xui»(l-X)u2)  >  Max  [Xf'  (uj.vj)  ♦  (1-X)  g’  (u2,v2)j 

0  <  V1  S.  U1 
0  <  v2  <  u2 


>  X  Max  g'iuj.Vj)  ♦  (1-X)  Max  g'(u2,v2) 

^  0  <  V]  <  uj  0  <  v2  -  u2 

>  Xfn<Uj>  ♦  (1-X)  fn(u2>  U?) 

The  foregoing  shows  that  if  rH(dR)  is  concave  in  dn,  then  fj(xj) 
is  concave  in  Xj.  It  further  shows  that  if  in  addition  Cn«i(*n  -  dR) 
is  concave  in  Xj^  -  dn,  then  fn(xn)  1®  concave  in  xR.  Thus,  by 


f 


mathematical  induction,  fR(xn)  as  defined  by  equations  (9)  and  (10)  is 
concave  in  xn  for  all  l  <  n  <  N. 

This  concavity  is  a  useful  property  because  it  simplifies  the 
determination  of  the  optimum  salvo  site,  dn*(xn)  when  applying  equation 
(10).  This  simplification  canes  about  in  finding  the  maximum  because, 
given  concavity,  any  local  maximum  is  also  a  global  maximum. 


Numerical  Example 

As  a  numerical  example  to  illustrate  the  application  of  equations 


(9)  and  (10),  lot 


N  -  6  passes 

xN  -  8  weapons 


sT  -  1.0 

Su  -  0. 98 


and  suppose  rH(dn)  has  the  form 


rH(dn)  -  V  (1.0  n)  ;  0  <  0  <  1.0 


Kor  this  example,  let 


y  -  1.0 


0  -0.84 


Throughout  this  work,  considerable  use  will  be  made  of  this 
form,  i.e.,  equation  (18),  for  the  salvo  effectiveness  function.  Note 
that  in  equation  (18),  rjj(dR)  is  a  monotonically  nondecreasing  concave 
function  of  dft.  This  functional  form  provides  a  two  parameter  family 
of  functions  that  can  be  used  to  approximate  a  considerable  variety  of 
possible  salvo  effectiveness  functions.  The  form  is  appropriate 
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because  salvo  effectiveness  functions  are  monotonlcally  nondecreasing 
functions  and  they  tend  to  be  concave,  l.e.,  salvo  effectiveness  ln- 
creases  monotonlcally  with  salvo  size,  but  there  tends  to  be  a 
diminishing  return  with  increased  salvo  size.  Furthermore,  this  form 
will  accomodate  the  case  where  we  seek  to  kill  the  target  and  lndi- 
vldual  members  of  the  salvo  are  assumed  to  be  delivered  independently. 
For  this  latter  case,  B  is  interpreted  as  the  probability  that  the 
target  survives  a  single  weapon  and  4*  -  1.0. 

Figure  3  illustrates  the  rH(dn)  function  and  Table  1  shows  the 
results  of  carrying  out  the  calculations.  From  this  table,  we  can 
read  the  optimum  allocation  of  weapons  among  the  passes  and  the  maxi¬ 
mum  expected  hits  for  any  initial  bomb  load  up  to  eight  and  for  any 
limiting  nisnber  of  passes  up  to  six.  Given  eight  bombs  (xN  -  8)  and  a 
maximum  of  six  passes  (N  »  6),  ehen  the  expected  hits,  f6(8),  Is  1.179, 
The  optimum  allocation  of  weapons  is  (d$*,  dj*,  d^*,  dj*,  d2*,  dj*)  - 
(2, 2, 1,1, 1,1).  It  Is  satisfying  to  note  that  for  any  given  n,  fn(*n) 

Is  an  Increasing  function  of  x„  and  for  any  given  xn,  fn(*n)  Is  an 
Increasing  function  of  n  as  would  be  expected. 

It  is  Interesting  to  note  that  fifty-three  seconds  were  required 
for  a  Fortran  IV  program  on  an  IBM  7096  to  generate  120  such  tables 
with  M  -  10  and  x^  ■  12.  The  concavity  property  was  not  used  In  this 
program,  i.e.,  complete  emraeration  was  carried  out.  Accordingly,  the 
running  time  could  probably  be  reduced  by  using  the  concavity  property. 
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Non  Recursive  Form 

Insight  can  be  gained  by  stating  the  problem  In  non  recursive 
form.  Let  Rn(dn,  dn.j,— »,di)  denote  the  expected  hits  obtainable  on 
passes  n,---,l  for  a  given  allocation.  The  constraints  stated  in 
equations  (l),  (2),  and  (3)  must  be  met.  With  one  pass  remaining,  we 
have 


Rl  (d  i )  ■  S^. 


With  two  passes  remaining. 


(19) 


1*2 (d 2 »  d^)  ■*  s^rH(d2)  ♦  Su  Rj(d^) 

•  Sfr^(d2)  ♦  S-j-^  Su  tjj(dj) 


(20) 


With  three  passes  remaining. 


^3 (d 3 *  d2»  d3)  «  Sj  r^(d^)  ♦  S'p  Su  R2(d2»  dj) 

-  ST  rH(d3)  ♦  St2  su  rH(dz)  +  ST3  Su2  r^)  (21) 


With  N  passes  remaining. 


,N-n 


%Wn»"""»^1^  “  Sf  £  (Sf  Sy)  ^H^n^ 
n»l 


(22) 


Thus,  the  original  optimization  problem  might  be  stated  as 


follows: 


m  Max  Sj  £  (Srj  Sy)  rjj(dn) 

dN,— ,di  n-1 


(23) 


i  ! 
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subject  to 
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N 

£  d  <  x, 

n  —  I 


(24) 


dn  i  0  J  1  <  n  <  N 


(25) 


This  problem  has  the  same  form  as  a  problem  of  sequential  allocation 
with  discounting.  The  product  Su  might  be  Interpreted  as  a  discount 
factor  as  used,  for  example,  by  Howard  (17). 


A  Parametric  Investigation 

We  have  seen  an  example  of  the  type  of  solution  that  Is  obtained 
for  given  values  of  the  Input  parameters.  It  Is  of  interest  to  see  how 
the  optimal  allocation,  (dN*,  djj- 1 *»“**'» dj*)*  and  the  maximum  expected 
hits,  C^j(x^) ,  vary  for  a  range  of  input  parameter  values.  This  Inves¬ 
tigation  can  be  facilitated  by  making  use  of  the  non  recursive  state¬ 
ment  of  the  problem,  equations  (23),  (24),  and  (25). 

Assume  that  rH(dn)  can  be  expressed  by  equation  (18).  Then 
equation  (23)  becomes 

fN<xN>  •  ST^[  Max  £  (ST  Su)N‘n  (l-e^)]  (26) 

d  i » - ,dN  n-1 


With  the  problem  expressed  In  this  form.  It  Is  clear  that  the  optimal 
allocation  for  given  H  and  xN  depends  only  on  the  two  quantities  (STSU) 
and  0  end  not  on  . 

Table  2  shows  the  variation  of  (d^*,  ,d^*)  versus 

(ST  Su)  and  0  when  xN  »  8  and  N  ■  8.  The  appropriate  salvo  effective¬ 
ness  functions  are  shown  in  Figure  3.  Note  that  because  of  the 
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discounting  structure,  if  >  n2 ,  than  dn*  >  dn*.  Thus  with  x^  ■  ft, 
there  will  never  be  any  reason  to  make  more  than  eight  passes  so  all 
the  (d^*,--- ,d j*)  In  Table  2  are  optimum. 

As  might  be  expected  for  given  0,  higher  survival  probability, 
l.e.,  larger  (ST  Su),  leads  to  more  passes.  Also,  for  given  (ST  Su), 
smaller  values  of  0  lead  to  more  passes.  This  is  reasonable  If  we 
view  0  as  controlling  the  rate  at  which  r^(dn)  approaches  Its  asymptotic 
level  'V  ;  see  equation  (18)  and  Figure  3.  If  0  Is  small.  It  means  that 
the  marginal  value  of  Increasing  dn  decreases  rapidly  as  dp  Increases. 
This  makes  small  salvo  size  and  a  corresponding  larger  number  of  passes 
more  advantageous  at  a  fixed  survival  level. 

It  Is  often  true  that  the  probability  of  survival  per  pass, 

(ST  Su),  Is  greater  than  0.98.  If  It  were  not,  a  sustained  air-to- 
ground  effort  would  probably  be  Impractical.  This  being  the  case,  unless 
0  is  close  to  1.0,  Table  2  indicates  that  the  optimal  allocation  tends 
to  be  an  even  distribution  of  weapons  among  passes.  The  case  where 
both  (ST  Su)  and  0  approach  1.0  enters  a  region  where  the  optimal  allo¬ 
cation  Is  quite  sensitive  to  both  (ST  Su)  and  0  as  is  Illustrated  by 
Tabie  2  but  Is  rether  uninteresting  otherwise  as  Is  borne  out  by 
Figure  A. 

Figure  A  shows  the  variation  of  fg(S)  versus  S  and  0  when 
S.j.  ■  *+'  •  1.0.  This  figure  gives  the  impression  that  as  the  value  of 
0  decreases,  fg(8)  becomes  more  sensitive  rc  the  value  of  Su.  This  is 
reasonable  because  as  0  decreases,  the  tendency  Is  to  moke  more  oasses, 
thus  making  survival  more  Important.  Based  on  the  sawe  tvpo  of  reason¬ 
ing,  it  Is  reasonable  for  fg(8)  to  be  more  sensitive  to  S(.  as  the  value 
of  Su  approaches  1.0. 
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Return  Versus  Attrition 

A  very  important  concept  will  now  be  introduced.  The  discussion 
so  far  has  considered  maximising  the  expected  hits  per  duel.  This  has 
been  referred  to  as  the  return.  The  example  of  Table  1  shows  that  the 
return  is  appreciably  higher  for  six  passes  than  it  is  for  only  or.e 
pass. 

The  foregoing  tells  only  part  of  the  story.  In  addition  to  the 
return,  the  cost  must  be  considered.  Cost  will  be  measured  in  terms 
of  aircraft  lost  c:  "attrition."  When  only  one  aircraft  is  Involved 
as  in  the  present  nlscusston,  expected  attrition  will  Indicate  the 
probability  of  the  aircraft  not  surviving. 

Note  that  for  a  complete  model,  the  cost  should  also  include 
other  factors  such  as  the  cost  of  weapons  and  the  cost  ot  fuel.  These 
will  be  excluded  here  since  they  are  often  negligible  compared  to  the 
cost  of  replacing  aircraft  and  pilots.  Further,  including  such  other 
costs  is  generally  a  straightforward  procedure.  For  the  simple  duel, 
the  maximum  expected  attrition  is  given  by 

Lq  -  1  -  (ST  SU)N  (28) 

Note  that  the  actual  expected  attrition  may  be  different  from  the  maxi¬ 
mum  expected  attrition  since  the  optimum  solution  may  call  for  less 
than  N  passes. 

Figure  5  shows  a  retum-versus-attriticn  function  for  the  ex¬ 
ample  of  Table  1.  This  figure  applies  to  the  case  where  t ix  weapons 
are  available  at  stage  N,  i.e.,  xN  -  6,  and  N  is  varied  from  1  to  6. 
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This  figure  shows  the  return  as  a  function  of  the  actual  expected 
attrition  because  in  the  example,  it  is  optimal  to  make  N  passes  if 
N  <  6. 

The  expected  attrition  entries  from  left  to  right  on  the  abs¬ 
cissa  scale  in  Figure  5  are  the  values  obtained  from  equation  (28) 
when  ST  Su  ■  0.98  and  N  -  1,2, 3,4, 5, 6,  respectively.  At  each  level  of 
expected  attrition,  the  return  is  the  maximum  expected  hits  assuming 
that  an  optimum  weapon  allocation  is  used.  For  example,  the  attrition 
level  of  0.06  results  when  N  ■  3.  We  see  from  Table  1  that  when  N  -  3 
and  -  6,  the  maximum  expected  hits,  f -j(6)  is  0.866.  Thus,  the 
ordinate  value  is  0.866  corresponding  to  an  expected  attrition  of  0.06 
in  Figure  5.  The  optimum  weapon  allocation  when  N  -  3  and  x^  -  6  can 
also  be  read  from  Table  1  as  (d3*,  d2*,  d^*)  -  (2,2,2). 

We  have  now  developed  a  model  for  optimising  the  simple  E^  duel 
and  shown  how  its  results  can  be  used  to  determine  the  retum-versus- 
attrltion  function.  We  will  proceed  to  complicate  and  modify  our  no¬ 
tion  of  a  duel,  but  the  return- vers us-attr It ion  function  will  be  a 
basic  characterizing  feature  of  all  the  duels  that  will  be  studied. 

In  addition  to  studying  other  duels,  determining  which  of  the 
points  on  tha  retum-versus-attrltion  function  corresponds  to  the  most 
desirable  weapon  allocation  will  also  be  the  subject  of  considerable 
subsequent  discussion. 


i 
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CHAPTER  IV 


GENERALIZING  THE  eh  DUEL1 

Incorporating  Probabilistic  Target  Acquisition 

We  continue  considering  a  multiple  pass  air-to-ground  attack 
where  the  return  is  in  terms  of  expected  hits  and  the  cost  is  In  terms 
of  the  number  of  aircraft  lost.  All  the  features  of  the  simple  EH  duel 
are  still  present  and  same  additional  factors  will  be  included. 

The  simple  E^  duel  assumes  the  weapons  allocated  to  a  pass  are 
sure  to  be  delivered  if  the  aircraft  survives  long  enough.  In  present 
close  support  operations,  the  presence  of  a  forward  air  controller  who 
directs  the  attack  generally  makes  this  a  valid  assumption.  Likewise, 
the  assumption  is  reasonable  for  many  interdiction  attacks  on  easily 
acquired  targets  such  as  bridges  or  harbor  facilities. 

There  are  also  interesting  situations,  however,  where  the  target 
is  not  easily  acquired  and  there  is  no  forward  air  controller.  This 
occurs  under  conditions  which  may  include  night,  bad  weather,  and 
obscu.a  targets.  The  ability  of  the  aircraft  to  acquire  targets  is 
related  to  its  sensor  capability  and  its  navigation  system.  Considera¬ 
tions  of  this  sort  are  becoming  increasingly  important  in  the  analysis 
of  weapon  systems  that  are  technologically  advanced. 

1It  may  be  useful  to  the  reader  to  refer  to  Appendix  A  either 
in  conjunction  with  or  after  reading  Chapter  IV.  That  appendix  indi¬ 
cates  how  the  problem  of  Chapter  IV  can  be  approached  by  first  making 
a  non  recursive  statement  of  the  problem  and  then  developing  the  re¬ 
cursive  relationships  in  a  manner  similar  to  that  of  Neshaos6r  (22). 
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Consider  the  following  abstraction  of  the  target  acquisition 
aspect  of  the  problem.  This  discussion  extends  the  general  discussion 
that  appeared  in  the  second  section  of  Chapter  I.  An  "initial”  pass 
is  made  when  the  target  has  not  been  acquired  previously  on  the  same 
sortie.  Ctice  target  acquisition  has  occurred,  additional  passes  on 
the  same  sortie  will  be  referred  to  as  "subsequent"  passes.  This  dis¬ 
tinction  allows  for  improved  target  acquisition  capability  once  the 
target  has  been  seen. 

When  an  initial  pass  is  made,  one  of  the  following  three  events 
will  occur: 

A0*  :  the  target  is  not  acquired 

AqD*:  the  target  is  acquired  but  weapon  delivery  is 

not  possible 

AJD:  the  target  is  acquired  and  weapon  delivery 

is  possible. 

•/f 

The  corresponding  events  for  a  subsequent  pass  are  symbolised  , 
AjD*,  and  A}D.  Since  weapon  delivery  may  or  may  not  be  possible  on 
pass  n,  transformation  of  the  state  variable  xn  is  probabilistic  and 
we  have  a  Markovian  decision  process. 

To  apply  the  recursive  relationship  that  was  developed  in 
Chapter  II  for  a  Markovian  decision  process,  equation  <II»18),  three 
levels  for  the  Markov  state  variable  1  are  defined  as  follows: 
i  -  1:  acquisition  has  not  yet  occurred 
1-2:  acquisition  has  occurred  and  no  weapons  were 

delivered  on  the  most  recenc  >*ass 
1  -  3t  acquisition  has  occurred  and  weapons  were 
delivered  on  the  most  recent  pass. 
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In  the  present  esse,  the  transition  probability  Is  Independent  of  the 
stage  n  decision,  dn,  so  we  shall  simply  denote  the  transition  proba¬ 
bility  as  P|j.  The  matrix  of  transition  probabilities  Is  given  In 
Figure  6  for  the  system  that  has  been  described. 


Transformation  of  the  state  variable  Xj,  depends  on  the  transi¬ 
tion  that  occurs  in  the  Markov  state  (analogous  to  equation  (II-9)). 

If  transition  is  to  Markov  state  1  or  2,  no  weapons  are  delivered.  If 
transition  Is  to  Markov  state  3,  dn  weapons  are  delivered,  hence  for 
all  1  <  n  <  N  and  1  <  1  <  3, 


xn-l  “  xn  »  J  "  1*2 

xn-l  ‘  xn"dn  5  i  m  3 


(1) 


It  can  be  seen  by  inspection  of  the  transformation  relation  equa¬ 
tion  (1)  and  considering  the  non  negativity  of  d,,  that  the  following 
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sets  have  the  required  property  that  x_f  S*  and  dn^sd  (Xn)  *or  a11 

n  n 

1  <  n  <  N,  1  <  1  <  3,  1  <  j  <  3. 

*  {  xns  xn  €-[o, - ,xN  ^  j-  <2) 

®dn(*n^  “  ^  dnJ  ^  ^  0,—  ,xn^^.  (3) 

Return  functions  that  have  the  required  aonotanicity  and  equl- 
valence  as  defined  In  Chapter  II  will  now  be  developed.  Weapon  delivery 
occurs  only  when  transition  Is  to  Markov  state  3  and  the  aircraft  sur¬ 
vives  to  the  point  of  weapon  release.  Thus ,  If  n  ■  1  and  for  all 
1  <  I  <  3, 

®1IJ  Cxl»  dl»  ^oj^xo^-l  "  ®  »  J  “  1*2 

«  ST  rH^dl^  »  J*3  (4) 

When  transition  Is  to  Markov  states  1  or  2,  no  weapons  are 
delivered.  If  represents  the  maximm  expected  hits  achievable 

in  n  passes  with  xn  weapons  and  in  Markov  state  1,  then  the  return 
associated  with  transition  to  Markov  states  1  or  2  is  th<*  probabll«tv 
of  surviving  pass  n  tines  fn_i  j  The  return  associated  with 

transition  to  Markov  state  3  is  the  probability  of  surviving  to  the 
point  of  weapon  delivery  times  the  expected  hits  achievable  by  the 
salvo  that  is  delivered  on  pass  n  plus  the  probability  of  surviving 
pass  n  times  £n_i  Thus,  if  n  >  1  and  for  all  1  <  i  <  3 

Snips*’  V  *n-l,J<*»*.l>3  -  ST  Su  ^-l.J^n-l5  5  J  -  1.2 

(5) 

*.  j  -  3 


ST  rH(dnJ  *  ST  Su  fn-l,3^xn-l^ 
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By  examining  aquations  (4)  and  (S)  It  la  claar  that  since  St> 
and  Su  are  non  negative  constants,  the  return  functions  have  the  re¬ 
quired  monotonicity  and  equivalence  properties.  The  sufficient  con¬ 
ditions  for  optimality  in  a  Markovian  decision  process  have  been  net. 
Recursive  application  of  equation  (11-18)  will  yield  that  are 

optimum. 

Replacing  che  various  parts  of  equation  (11-18)  by  their  equi¬ 
valent  expressions  from  equations  (1),  (3),  and  (4)  and  considering 
equation  (2)  gives  the  following.  If  n  -  1  and  for  all  1  <  i  <  3, 
xl€SXi> 


flt(xt)  -  Max  [p|3  ST  rH(4|)j 
0  <  dl  <  *1 


Assuming  that  rH(dn)  is  non  decreasing  in  dn,  this  becomes 


(6) 


fll<xlJ  "  pi3  ST  rH(xl* 


(7) 


If  we  replace  the  various  factors  of  equation  (1 1-18)  by  their 

equivalent  expressions  from  equations  (1),  (3),  and  (5),  and  consider 

equation  (2),  the  following  is  obtained.  If  2  <  n  <  N  and  for  all 

i  <  »  <  3.  *nesx  , 
n 

2 

{ni^xn^  *  Max  ^  pjj  ST  Su  fn.|  j(*n) 

0  £  dn  <  *n  i-1 


♦  P13  lsT  rH<dn>  ♦  ST  Su  -  d„)]  (8) 
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which  can  be  written  as 

2 

fnl(xn)  "  ST  Su  2  Pij  fn-l,J(xn> 

J-l 

♦  ST  pi3  Max  CrK<V  ♦  Su  fn-l,3<*n  *  dn>]  <9> 
0  <  dn  <  xn 

Equations  (7)  and  (9)  constitute  the  recursive  relationships 
that  will  yield  maximm  expected  hits  and  the  corresponding  optimum 
weapon  allocation  Cor  the  duel  where  target  acquisition  is  considered. 
The  optimum  salvo  size  is  a  function  of  weapons  available,  the  Markov 
state  of  the  system,  and  the  number  of  passes  remaining.  It  will  be 

lAr 

denoted  dnt(xn). 


A  Simplifying  Feature 

When  making  calculations  using  equations  (7)  and  (9),  the 
following  observation  is  helpful.  These  equations  depend  on  1  only 
because  of  pjj.  From  the  Markov  state  transition  matrix.  Figure  6, 

P2j  ■  Pjj  for  all  1  <  J  <  3,  Thus,  equations  (7)  and  (9)  have  the 
following  properties: 

fn2<*n>  "  fn3<*n>  <*<» 

dn2  <xn>  "  dn3  <*n>  <n> 

This  fact  allows  considerable  reduction  in  the  amount  of  computation 
required  to  evaluate  equations  (7)  and  (9).  It  also  eliminates  one* 
third  of  the  items  of  data  that  would  otherwise  have  to  be  Included 


when  tabulating  the  results. 


r 


Note  that  while  the  foregoing  observation  Is  a  convenience.  It 
does  not  allow  reduction  In  the  number  of  Markov  states.  States  2  and 
3  must  both  be  Included  because  transition  to  state  2  Involves  no 
weapon  delivery  while  weapon  delivery  does  accompany  transition  to 
state  3. 


Concavity 

The  concavity  property  that  v*s  proved  In  Chapter  III  for 
equation  (III-10)  also  holds  for  equation  <9).  in  chapter  III  we 
showed  that  If  r(dn)  Is  concave  In  dn  and  j)  Is  concave  In 

xn-l’  thon  the  expression 


0  ,  x  CrH<a")  *  S"  '  a")] 

u  i  an  i  xn 


Is  concave  In  xn.  Thus,  if  rH(dn)  and  *n.i ,j<xn-l)  conctv*  In 

dn  *n-l  respectively,  since  ST  >  0,  Su  >  0,  and  ptJ  >  0  for  all  l,j, 
^ni^n^  as  determined  by  equation  (9)  is  concave  In  x,j  for  1  <  1  <  3, 

2  <  n  <  N.  Further,  It  Is  clear  from  equation  (7)  that  If  ru(x.)  Is 
concave  In  Xj,  then  fj^Xj)  is  concave  In  x1  for  all  Xjg  Sx  ,  1  <  i  <  3. 
Thus,  by  mathematical  Induction,  fni(xn)  Is  concave  In  Xj^  for  all 
l<n<N,  1  <  i  <  3,  xn£  Sj^.  This  property  can  be  used  to  reduce  the 
amount  of  calculation  that  is  required  because  It  means  that  any  local 
maximum  that  is  found  for  equation  (9)  is  also  a  global  maximum. 

Incorporating  Multiple  Modes  of  Attack 
We  will  now  Introduce  Into  the  model  the  notion  that  the  pilot 
need  not  make  every  pass  in  exactly  the  same  way.  In  addition  to 
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■sleeting  the  optimum  salvo  site,  he  must  specify  other  quantities 
such  as  direction  of  approach,  speed,  dive  angle,  and  pullout  altitude. 
The  aggregation  of  values  taken  by  these  other  quantities  on  a  given 
pass  will  be  referred  to  as  the  "mode  of  attack." 

Let  the  discrete  variable,  1^,  be  the  mode  of  attack  index  at 
pass  n  and  let  Indicate  the  set  of  all  values  that  can  take. 

Thus  at  each  pass,  values  for  the  two  decision  variables  dn  and  kn 
must  be  chosen.  These  decision  variables  are  the  elements  of  the  de¬ 
cision  vector  Dn  -  (dn,  1^). 

For  the  most  part,  the  modlf tcatlcns  required  to  accommodate 
mode  of  attack  are  minor.  A  number  of  the  quantities  In  the  model  de¬ 
pend  on  the  value  of  kn.  The  survival  probabilities  become  ST(kn)  and 
Syfk^).  The  Markov  state  transition  probabilities  become  PijO^). 

The  salvo  effectiveness  function  becomes  rH(dR,  kn)  •  rH^Dn^»  ch® 
expected  hits  per  salvo  versus  salvo  size  and  mode  of  attack. 

One  nontrivial  modification  of  the  model  is  required.  In  the 
duel  with  probabilistic  acquisition,  we  dealt  at  each  stage  with  a 
functional  equation  of  the  form 

*  Max  J  L  Pij  gnij  [xn,  d„,  fn.1>J(xn.l)] 
dnCSdn(Xn)  I  j-1 

The  expression  In  equation  (12)  represents  the  expected  hits 

for  given  n,  1,  Xjj,  and  dn.  Since  this  expression  represents  the 
oxpected  hits  for  a  given  value  of  n,  and  since  ST  and  Su  arc  con¬ 
stants,  the  probability  of  surviving  the  remaining  n  passes  is  at  least 
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(ST  Su)n  tor  all  dnGSd^(xT1),2  In  effect,  the  dn*  (x^  that  satisfy 
equation  (12)  maximise  the  expected  hits  subject  to  the  constraint  that 
the  expected  attrition  cn  the  remaining  n  passes  does  not  exceed 
1  -  (ST  Su)n,  i.e.,  not  more  than  n  passes  are  to  be  made  and  the  prob¬ 
ability  of  surviving  each  pass  is  (ST  Su). 

When  multiple  nodes  of  attack  are  available  at  each  pass,  we 
must  deal  with  a  functional  equation  of  the  more  general  form 

c  3  -i 

fni<XV  *  i  E  Pij^  «rlj  CXA,  Dn.  f,,.! 


Equation  (13)  is  the  same  as  the  corresponding  equation  (11-18)  except 

for  having  recognised  chat  the  Markov  state  transition  probabilities 

depend  only  on  the  kn  component  of  Dn.  Note  that  the  residual  state 

vector  X'  now  appears  in  place  of  the  variable  x_  which  appeared  in 
n  11 

equation  (12). 

The  expression  {•}  in  equation  (13)  represents  the  expected 
hits  for  given  n,  i,  X^,  and  Dn.  The  approach  at  each  stage  will  be  to 
select  Dn*l  (X^)  so  as  to  maximise  the  expected  hits  subject  to  a  con¬ 
straint  on  the  expected  attrition.  Let  s,,  denote  the  constraining 
probability  of  surviving  the  remaining  n  passes.  A  complete  descrip¬ 
tion  of  the  state  of  the  system  at  pass  n  now  requires  knowledge  of  the 


4Note  that  Sj  and  Su  could  depend  on  n  and  these  comments  would 
still  apply.  The  constraining  value  of  the  expected  attrition  for  the 
remaining  n  passes  would  then  be 

n 

1  -  tt  Sf(n')  Su(n’)  ,  where  n*  is  a  dumsy  variable. 
r!~l 


I'rinHWmaH*,  -im 
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value  of  the  Markov  state  variable,  1,  the  nunber  of  weapons  remaining, 

3^,  and  the  constraining  probability  of  surviving  the  remaining  n  passes, 
3 

sn.  The  residual  state  vector  is 


X*  -  (x  ,  s  ) 
n  n*  n 


(14) 


Now  consider  the  various  parts  of  equation  (13)  as  they  apply  to 
this  model.  The  definitions  of  the  Markov  states  are  unchanged.  The 
Markov  state  transition  probabilities  are  given  by  Figure  6  except 
that  all  of  the  acquisition  probabilities  now  depend  cm  kn. 

The  transition  equations  for  the  state  variables  are  as  follows. 
For  all  1  <  n  <  N,  1  <  1  <  3 


*n-l  *  *n  J  i  -  I*2 

xn-l  ■"  *n  “  dn  5  j  -  3 


(15) 


n- 1 


sT(kn)  Su(kn) 


1  <  J  <  3 


kn«s 


kn<*n> 


(16) 


where  sc  »  1.0.  Equations  (15)  are  the  same  as  equation  (1). 

Equation  (16)  can  be  rationalized  as  follows.^  Let  sR  be  the  actual 
probability  of  surviving  the  remaining  n  passes.  Thus,  we  require  that 
>  sn  ;  1  <  n  <  N  (17) 


•» 

Note  that  in  equation  (12),  sn  has  a  single  value  for  given  n, 
i.e. ,  it  takes  the  value  1  -  (ST  Su)n.  Thus  in  equation  (12),  it  is 
not  necessary  to  explicitly  state  the  value  of  sn  when  defining  the 
state  of  the  system  for  given  n. 

^Equation  (16)  also  follows  from  the  expression  of  the  proba¬ 
bility  of  survival  constraint,  equation  (A-7),  that  is  given  in 
Appendix  A. 


Now  if  s0  -  1.0  and  n'  is  a  dunray  variable. 


V.  -  n  ST(kn.)  SyOcj,! )  -  S.^)  Su(k„)  s,^  ;  1  <  n  <  N  (18) 

n'-l 


So  using  equation  (17), 


8n  —  3n  "  sT^Si)  8n-l 


/\  _ sn 

Sn-1  “  ST(kn^  StJ(kn) 


By  inspection  of  equations  (15)  and  (16)  and  considering  the 

non  negativity  of  dR,  we  can  define  the  following  sets  which  have  the 

property  that  J^CS*  ,  d_eSd  (x,^,  s„«Ss  ,  and  k^S*  (sn)  for  all 
n  n  n  n 

1  <  n  <  N. 

s*„  '  {*»•  x"e  W . *»}}  <n) 

Ssn  -  { sn:  3N  <  8n  <  >  }  (22) 

sdn<xn>  -  {dn!  dn€-[°»1»—»xn]-|  <23> 

skn<*n>  -  {  V  W  Vkr.>  >  8n»  (26) 

where  xN  >  0  and  0  <  s^  <  1.0,  The  quantity  sN  is  the  smallest  value 

of  Che  constraining  survival  probability  that  is  of  interest.  It 
establishes  the  range  of  values  of  over  which  solutions  will  be 
obtained . 


— 1  — ‘rieiijH 
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Th«  return  functions  are  entirely  analogous  to  those  for  the 
duel  with  probabilistic  acquisition,  equations  (4)  and  (5).  If  n  •  1 
and  for  all  1  <  1  <  3, 

gnj  Lxi*  °i»  co,j  "  0  ;  J  “  i»2 

-  ST(kj)  ^(dpkj)  ;  j  -  3  (25) 

If  2  <  n  <  N  and  for  all  1  <  I  <  3, 

®nij  C*n’  Dn»  fn-l,J^*n-l^ 

-  ST(kn)  SyO^)  fn.itj(xn-i.sn.i)  *  )  -  1,2 

*  St«Si)  rH^dn,kn^  *  sT^n^  Su^kn^  fn-l, j ^xn-l*8n-l^ 

;  J  -  3  (26) 

Now  replacing  the  various  parts  of  equation  (11-18),  or 
equation  (13),  by  their  equivalents  from  equations  (15),  (16),  (23), 
(24),  (25),  (26),  and  considering  equations  (21)  and  (22)  gives  the 
functional  equations  for  the  duel  with  acquisition  and  multiple  attack 
nodes.  If  n  »  1,  1  <  l  <  3, 

fli(x1,6i)  «■  Max  E  Pl3^kl^  sx<kl)  rH*dl>  kP 

KlSS^S!) 

dl€sd1<xl> 

-  Max  [Pi3<kl>  ST(kl)  rH(xlf  kx)]  (27) 

kl€Skl<8l) 

since  rR(dj,  kj)  Is  assumed  to  be  a  monotonlcally  non  decreasing 
ftaictlon  of  dj. 


r^rrr»ar.ffTf  •'"'War*  '  f ’<!  IWflf*  IW* 4'  IfJT1^ 


f 


•  i 

\ 

f 
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If  2  <  n  <  N,  I  <  I  <  3, 


Cni(xn»sn)  “  **« 

*n«sk. 


(sn){>l  PiJ(kn)ST(kn>su<kn)fn-l,J  (X«’sT<kn)su(kn)) 

(28) 


dn«Sd  <*n> 


♦  P|3<kn>  [sx(kn)rH<dn,ki|)  ♦  ST(kn)Su(^)fn.1|3  J 

Since  the  first  tens  of  ^  In  equation  (28)  Is  Independent  of  d^ 


we  can  write 


■  s,^<5n,{ST<K">s”<K”>  A 

(29) 

♦  q  <  [rH<dn*kn>**u<kn>fiiPl,3  (*n^n»5^^^^ 


The  optimal  values  of  the  decision  variables  will  be  denoted  d  *(3^,8^ 
and  hni^.Sn)- 

For  the  same  reasons  that  were  mentioned  with  respect  to  equa¬ 
tions  (7)  and  (9),  equations  (27)  and  (29)  have  the  following  pro¬ 
perties  : 


fn2<xn*sn)  “  fn3<*n»sn> 


drt2<*n*V  -  dn3<*n»*n> 


kn2<xn»Bn>  -  kn3(*n»,n> 


(30) 

(31) 

(3?) 
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Using  a  Discrete  Attrition  Constraint 

Before  considering  an  example  of  the  application  of  equations 

(27)  and  (29),  a  practical  difficulty  must  be  faced.  The  quantity 

has  been  created  as  a  continuous  variable.  Also,  S„(k  )  and  S  (k  ) 

T  n  u  n 

can  take  any  value  from  0  to  1.0.  Under  these  assumptions,  the  de¬ 
velopment  Is  rigorous. 

In  making  numerical  calculations,  however,  sn  cannot  be  treated 

as  a  continuous  variable  when  fn|(xn,sn)  Is  tabulated.  Accordingly, 

the  Interval  £sjg,  l.oj  will  be  divided  into  a  discrete  number,  M,  of 

Increments  of  size  ^  s.  At  each  stage,  sn  is  treated  as  a  discrete 

variable  taking  only  the  values  sN  +  nAs  where  m  -  0,1,2,— ,M. 

This  constitutes  a  modification  of  the  definition  of  S  ,  equation  (22). 

The  set  S_  will  now  be  def  ned  as 
“n 

Ssn  "  {sn:  sn  €{sN»  sn  *  ^s*  sN  ♦  2  As,—  ,s„  +  ;  1  <  n  <  N 


We  must  also  modify  the  transformation  equation  (16)  because  the 

quotient  sn/ST(kn)Su(kn)  will  not  In  general  produce  a  value  of  sn.j 

such  that  3  i € S-  as  defined  by  equation  (33),  Our  approach  is 
*  n-1 

to  select  the  next  larger  acceptable  value  for  sn_j  or  symbolically, 


°n-l  "  1 


Vkn)Su(kn) 


A  s  ;  1  <  J  <  3  (34) 


kn€Skn(sn) 


whore  the  symbol  (x)  means  the  largest  Integer  value  no  greater 
than  X.  The  definition  of  Sk^(sn),  equation  (24),  still  applies. 


•UL4W1  II'  Jdi-::  MSN  m*  ■mu Jaw.. 


rw,^nf— ""TT'iwv**-'’ 
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To  demonstrate  that  equation  34)  will  satisfy  the  survival 
constraint,  write  equation  (34)  as 


ST<kn)Su(kn) 


Removing  the  largest  Integer  value  restriction,  equation  (35)  becomes 


Sf(kn)Su(kn)  >  *  "  sn-l 


which  becomes 


sn  £  ST<kn>Vkn>  sn-l 


Vl"  WW 


Thus,  the  sn_j  that  is  produced  by  equation  (34)  will  always  be  at 
least  as  great  as  the  sn_j  that  is  produced  by  equation  (16). 

It  Is  well  to  note  that  by  admitting  only  certain  values  of 
sn,  we  have  a  more  restrictive  optimisation  than  woulo  result  if  sn 
could  be  continuous.  Thus,  the  maximum  expected  hits  obtained  from 
the  discrete  case  cannot  be  greater  than  the  maxlmim  expected  hits 
obtainable  if  sR  could  be  treated  as  continuous.  The  amount  of  dls- 


crepancy  depends  on  the  value  of  As.  For  a  given  problem,  a 
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sensitivity  analysis  would  be  desirable  to  determine  the  extent  to 
which  the  solution  depends  on  the  value  of  As,^ 


Numerical  Example 

to  illustrate  the  application  of  equations  (27)  and  (29), 
suppose  an  aircraft  has  eight  boobs  and  can  make  a  maximum  of  three 
passes  at  the  target. 

N  -  3  passes 
X||  —  8  bombs 

Suppose  further  that  acquisition  is  probabilistic  on  each  pass  with 
probabilities  as  shown  in  Table  3  and  the  corresponding  matrix  of 
Markov  state  transition  probabilities  as  shown  in  Figure  7  for  all  k,^. 
This  transition  matrix  can  be  computed  from  the  values  In  Table  3  and 
by  use  of  the  Markov  state  transition  probabilities  in  Figure  6. 


5It  is  also  possible  to  use  an  additive  transformation  relation¬ 
ship  for  the  attrition  constraint  if  Sjj,  Sxd^),  and  SuO^)  are  close 
to  1.0.  Suppose  A(kn)  2  1.0  -  SjO^)  Su(kn).  Then  equation  (16)  can 
be  written 

s  <*>  i 

3n’1  "  ~  A(i4)  "  *"  if0  ^A05n>^  *  *n  *  *n  Mkn) 

If  A(kn)  Is  small.  Further,  if  ^  is  close  to  1.0, 

*n-l  “  ®n  ♦  A(kn) 

which  might  be  used  in  place  of  equation  (34).  Now  if  S^Oc,,)  and  SjjOc,,) 
are  allowed  only  to  take  values  such  that  A(kn)  ■  nas  where  n  is  a 
positive  integer  and  A  s  is  as  used  in  equation  (33),  then  use  of  the 
above  additive  survival  constraint  transformation  avoids  the  truncation 
error  that  is  Introduced  by  equation  (34).  Where  it  is  applicable, 
this  procedure  may  be  preferable  to  the  use  of  equation  (34)  because 
the  implications  of  restricting  the  values  that  the  Input  parameters 
can  take  are  perhaps  easier  to  understand  than  the  implications  of  the 
truncation  that  occurs  in  equation  (34).  This  is  especially  true  be¬ 
cause  the  truncation  error  tends  to  be  cumulative  as  successive  trans¬ 
formations  are  performed. 


! 
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TABLE  3 

EXAMPLE  ACQUISITION  PROBABILITIES 
(All 


P  U0) 

m 

.725 

P  (A0 P) 

- 

.340 

P  (Ai) 

- 

.915 

P  (AjD) 

- 

.775 

i  -  1:  A* 

1-2:  AD* 

1-3:  AD 
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J  -  1 

J  -  2 

J  -  3 

A* 

AD* 

AD 

0.275 

0.385 

0.340 

0 

0.225 

0.  775 

0 

0.225 

0.  775 

Fig.  7. Example  values  of  Markov  state 
transition  probabilities. 


Let  there  be  four  modes  of  attack  available  on  each  pass  where 
variation  In  mode  of  attack  does  not  affect  the  acquisition  probabili¬ 
ties  but  does  have  effect  on  both  survival  probabilities  and  salvo 
effectiveness.  Suppose  S^>(kn)  -  Su(kn)  for  all  modes  and  for  all 
1  <  n  <  N.  The  values  of  the  survival  probabilities  are  given  in 
Figure  8.  It  is  also  convenient  to  note  the  attrition  per  pass  that 
is  defined  in  the  insert  on  Figure  8. 


1  'rtta’d*  hlNiaMKiT1  ,•  \« 


■ump.  in»w.  |f  nr*«  w  i  ww>  w  nn*  wr  "in" t ■  wwww"« wir"  »nfr,n»w.wnrnPW^|w,?in 
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Salvo  ef fectlveness  versus  salvo  size  and  node  of  attack  is 
given  In  Figure  9.  These  curves  were  generated  by  use  of  equation 
(Ill- 18).  The  parameters  H'  and  8  have  the  sasie  meaning  they  did  in 
Chapter  III.  Note  that  in  Figure  9J  ¥  Is  held  constant  and  different 
salvo  effectiveness  functions  are  obtained  by  using  different  values 


As  the  mode  of  attack  index  (kn)  Increases  from  1  to  U ,  the 
survival  probabilities  get  progressively  higher  while  the  salvo  effec¬ 
tiveness  gets  progressively  lower.  This  type  of  situation  might  arise 
if  increasing  values  of  kn  correspond  to  increasing  aircraft  speed. 

Calculations  for  this  example  were  made  with  xN  -  8  and  sN  m 
0.976.  weapons  are  assumed  to  be  allocated  in  groups  of  one.  The 
survival  constraint  was  varied  in  increments  o(  As  •  0.002. 

The  principal  results  are  tables  of  fni(xn»*n)*  t,nT^xn»8n^*  *nd 
knj(xn»an).  The  tables  that  were  generated  for  this  example  provide 
the  foregoing  information  for  any  number  of  weapons  remaining  up  to 
eight,  for  any  attrition  constraint  up  to  .02 6  and  for  any  number  of 
passes  remaining  up  to  five.  Tables  6,  5,  and  6  are  extracts  taken 
from  these  tables  at  n  -  3.  The  complete  tables  are  contained  in  an 
unpublished  computer  printout  that  Is  currently  in  the  possession  of 
the  author. 


For  a  given  n,  each  of  these  tables  has  three  variables.  The 
survival  constraint  is  stated  In  terns  of  attrition  (I  -  s_).  There 


are  two  entries  in  each  block  of  each  table.  The  first  entry  applies 
to  Markov  state  1  (target  not  yet  acquired)  and  the  second  entry 
applies  to  Markov  states  2  and  3  (target  acquisition  has  occurred). 


WM,' 


Salvo  effectiveness  versus  salvo  size  and  node  of  attack 


f 
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TABLE  4 


MAXIMUM  EXPECTED  HITS!  f31(x3,33> 


Number  of  Weapons 

Remaining 

(x3)  ^  ' 

4 

6 

8 

|  .006 
$ 

.226 

.515 

.274 

.624 

.301 

.687 

S'rt 

5  “  .  012 
g- 

«r<  "W* 

,562 

.694 

.709 

.908 

.850 

1.066 

■u 

£  .018 

JJ 

< 

.724 

.888 

.929 

1.124 

1.093 

1.301 

.024 

.873 

1. 014 

1.089 

1.278 

1.246 

1.509 

TABLE  5 

BEST  SALVO  SIZE*  d3* 

(X3,83) 

Number  of  Weapons 

Remaining  (x3) 

4 

6 

8 

w  .006 

4 

6 

8 

e 

4 

6 

8 

*2  ~ 

5  «  .012 

2 

3 

5 

1 

2 

3 

4 

L 

£  .018 

2 

2 

3 

4J 

«< 

2 
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TABLE  6 

BEST  MODE  OF  ATTACKS  k3*(x3,s3) 

Number  of  Weapons  Remaining  (x3) 


8 


3 

3 


4 

3 


4 

3 


3 

2 


Examination  of  Table  4  shows  that  the  maximum  expected  hits 
Increase  with  number  of  weapons  available  at  a  fixed  attrition  con¬ 
straint  and  It  increases  as  the  attrition  constraint  Is  relaxed  with 
a  fixed  number  of  weapons  available.  Also,  the  maximum  expected  hits 
Is  greater  when  target  acquisition  has  occurred  (Markov  states  2  and  3) 
than  when  target  acquisition  has  not  yet  occurred  (Markov  state  1). 

All  these  trends  agree  with  Intuition. 

Table  3  indicates  the  best  salvo  size  as  a  function  of  the  state 
of  the  system.  Note  that  these  results  indicate  the  number  of  weapons 
to  deliver  if  target  acquisition  and  weapon  delivery  occurs.  If  tar¬ 
get  acquisition  and  weapon  delivt  does  not  occur,  no  weapons  are  to 
be  delivered.  From  Table  5,  the  beat  salvo  size  increases  as  the 
niaaber  of  available  weapons  increases  with  a  fixed  constraint  on  th . 
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expected  attrition  and  decreases  as  the  constraint  on  the  expected 
attrition  becomes  less  restrictive  with  a  fixed  nimbei  of  weapons 
available.  These  trends  agree  with  intuition. 

By  using  the  series  of  tables  from  which  Tables  5  and  6  were 

£ 

extracted,  a  complete  policy  for  the  duel  can  be  constructed.  Such  a  | 

policy  is  shown  in  Figure  10  for  the  example  of  Tables  4,  5,  and  6. 

Figure  10  shows  three  columns  of  blocks.  The  left  hand  colioan 
applies  when  N  -  i  passes  remain,  the  second  column  of  blocks  applies 
when  n  «  2  passes  remain  and  the  third  column  of  blocks  applies  when 
n  -  1  pass  remains.  Each  block  contains  three  entries  on  the  left 
which  define  a  state  of  the  system.  The  entries  on  the  right  in  each 
block  indicate  the  best  action  corresponding  to  the  state  that  Is  Indi¬ 
cated  by  the  entries  on  the  left  In  the  same  block.  The  appearance  of 
the  symbol  A  Indicates  that  target  acquisition  has  occurred  and  the 
symbol  A*  indicates  that  target  acquisition  has  not  occurred.  The 

i 

arrows  connecting  the  blocks  show  the  possible  transitions  from  state  j 

to  sta’-e. 

/  j 

f  Figure  10  shows  the  policy  for  che  case  where  the  constraining 

j 

value  of  expected  attrition  ac  Che  beginning  of  the  duel  is  1  -  s-j  -  > 

O.ol2.  When  the  first  pass  is  made,  acquisition  has  not  yet  occurred  J 

ii 

and  eight  weapons  are  available.  The  policy  is  to  make  the  first  pass 
in  mode  U  and  deliver  five  weapons  if  target  acquisition  and  weapon 
delivery  occurs.  If  target  acquleltlon  falls  to  occur  on  the  first 
pass ,  then  the  system  ie  in  state  A*,  X£  -  8,  and  1  -  Sj  ■  .^08  with 
2  passes  remaining;  the  best  action  is  to  use  mode  4  with  the  intention 
of  delivering  five  weapons.  If  targot  acquisition  occurs  but  no  weapon* 


home 


t 


are  delivered  on  the  first  pass,  the  system  is  in  state  A,  x2  ■  8,  and 
1  -  S2  *  .008  with  two  passes  remaining;  the  best  action  is  to  use 
mode  4  with  the  intention  of  delivering  five  weapons.  If  target  ac¬ 
quisition  and  weapon  delivery  occurs  on  the  first  pass,  then  the  system 
is  in  state  A,  x2  -  3,  and  J  -  Sj  -  .008  with  two  passes  remaining;  the 
best  action  is  to  use  mode  2  with  the  intention  of  delivering  three 
weapons . 

The  following  are  some  further  comments  about  the  results  of  the 
foregoing  example.  The  decrease  in  best  salvo  size  with  less  restric¬ 


tive  attrition  constraint  for  fixed  number  of  weapons  available  is  an 
Interesting  phenomenon.  We  might  think  of  the  less  restrictive  attri¬ 
tion  constraint  as  representing  a  less  conservative  attitude,  l.e., 
more  willing  to  take  a  chance.  Now  as  our  attitude  becomes  less  con¬ 
servative,  we  are  less  concerned  about  survival  and  more  concerned  about 
maxlmua  effectiveness.  The  most  conservative  thing  to  do  is  to  deliver 
all  weapons  on  the  first  pass.  As  our  attitude  becomes  less  conserva¬ 
tive,  we  depart  further  and  further  from  this  policy,  i.e.,  the  salvo 
sizes  at  a  given  pass  become  smaller  thus  leaving  more  weapons  for 
future  passes.  As  the  attitude  becomes  less  and  less  conservative, 
there  comes  a  point  where  we,  in  effect,  ignore  survival  altogether 
and  simply  maximize  effectiveness.  Relaxing  the  attrition  constraint 
beyond  this  point  would  have  no  further  effect  on  tha  policy  relating 
to  salvo  size.  Note  that  the  djjixj.sj)  values  in  Table  5  are  begin¬ 
ning  to  reflect  this  phenomenon  since  the  optimum  salvo  size  is  nearly 
the  same  at  1  -  s-i  -  .018  and  1  -  s->  •  .024. 
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As  a  further  comment,  we  note  in  Table  6  what  seems  to  be  a 
general  outcome  that  Is  illustrated  by  this  example.  The  best  mode  of 
attack  is  more  conservative  when  acquisition  has  not  yet  occurred,  i.e., 
the  first  entry  in  each  block  in  Table  6  not  less  than  the  second  entry 
in  the  block.  This  reflects  the  fact  that  the  acquisition  probabilities 
do  not  depend  on  mode  of  attack  in  this  example  while  the  survival  prob¬ 
ability  and  weapon  effectiveness  do  depend  on  mode  of  attack.  When 
acquisition  has  not  yet  occurred,  the  tendency  seems  to  be  to  use  a 
"safer"  mode  of  attack  and  locate  the  target  in  order  to  save  the  air¬ 
craft  for  a  less  conservative  attack  on  a  subsequent  pass  when  the 
target  has  already  been  acquired.  Cnee  acquisition  has  occurred,  we 
tend  to  use  the  more  effective  mode  of  attack. 

Having  developed  a  generalized  E„  duel  and  discussed  some 
example  results,  we  will  close  this  chapter  by  noting  that  as  with 
other  duels  the  return-versus-attrltion  function  is  an  Important 
characterising  feature  of  the  duel.  Figure  11  shows  this  function  for 
the  foregoing  example.  The  data  for  this  figure  were  taken  from  the 
same  table  that  the  data  tor  Table  U  were  taken  from.  Figure  11  as- 
stages  that  acquisition  has  not  yet  occurred  (Markov  state  1  »  l), 
eight  weapons  are  available,  and  threo  passes  can  be  made.  A  particular 
selected  policy  like  that  of  Figure  10  applies  to  each  of  the  points  on 
the  retum-versus-attritlon  function. 
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CHAPTER  V 


A  DUEL  WITH  A  DIFFERENT  OBJECTIVE 
The  PR  Duel 

Now  consider  an  air-to-ground  duel  in  which  the  objective  Is  to 
hit  Che  target  at  least  once.  This  type  of  situation  might  arise  in 
an  attack  on  a  small  well  fortified  position  requiring  a  direct  hit. 

A  near  miss  is  assumed  to  do  no  damage  and  one  hit  is  assumed  to  be 
adequate.  In  this  discussion,  the  return  will  be  referred  to  as 
"probability  of  kill"  and  the  cost  is  in  terms  of  the  number  of  air¬ 
craft  lost.  This  will  be  referred  to  as  the  "PK  duel.”  The  model  to 
be  developed  is  quite  similar  to  the  model  that  was  developed  in 
Chapters  III  and  IV. 

Let  us  first  study  a  simple  duel  analogous  to  that  of  Chapter 
III.  In  a  simple  PK  duel  where  probability  of  kill  is  the  objective, 
the  stage  return,  18  the  Probablllty  Kill  versus  salvo  site. 

The  quantity  fn(xn)  becomes  the  maximum  probability  of  kill  achievable 
with  n  passes  and  xR  weapons  remaining. 

The  state  variable  transformation  is  given  by  equation  (III-3). 

The  sets  Sx  and  Sj  (x„)  ere  defined  by  equations  (III -A)  and  (III-5), 
n  n 

The  return  functions  for  this  case  are  somewhat  different  from 
those  of  the  model  for  deaLing  with  expected  hits.  When  one  pass  re¬ 
mains  and  dt  weapons  are  to  be  delivered,  the  probability  of  kill  is 
the  probability  of  reaching  the  point  of  delivery  times  the  probability 
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Chat  the  salvo  scores  at  least  one  hit.  Left  over  weapons  have  no 
value.  Thus,  If  n  -  1, 

g1[xl,  fo(x0)]  -  ST  (1) 

When  n  passes  are  yet  to  be  made,  the  composition  of  the  maximum  n-1 
stage  return,  *  with  the  stage  n  return,  r|((dn),  can  be 

accomplished  as  follows. 

Suppose  dn  weapons  are  to  be  delivered  on  pass  n  and  the  re¬ 
maining  xn  -  dn  weapons  are  to  be  delivered  on  the  remaining  n- 1  passes. 
Target  kill  can  occur  only  once  and  It  can  occur  In  one  of  two  mutually 
exclusive  ways.  The  target  can  be  killed  on  pass  n,  or  target  kill  can 
occur  on  one  of  the  remaining  n-1  passes.  Since  pass  n  chronologically 
precedes  the  remaining  n-1  passes,  If  the  target  Is  killed  on  pass  n, 
it  cannot  be  killed  on  one  of  the  n-1  remaining  passes.  The  probability 
of  target  kill  occurring  on  pass  n  is  STrK(dn).  The  probability  that 
the  target  Is  not  killed  on  pass  n  and  that  it  is  killed  on  one  of  the 
remaining  n-1  passes  Is  ST  S^l-r^d,,)]  fn_i(xn.i).  Since  these  two 
modes  of  target  kill  are  mutually  exclusive,  we  get  the  n  stage  proba¬ 
bility  of  kill  by  adding  the  two  terms.  Thus  for  all  2  <  n  <  N, 

®n  Cxn’  ^n*  ^n-l^Si-l^ 

•  ST  rK(dn)  ♦  ST  Su  [l  -  rK(dn)]  fn.^Xn.j)  (2) 

By  examining  equation  (1),  we  see  that  If  rK(dj)  Is  defined  for 
all  XjC  Sx^  and  djCSd^(xj),  then  the  return  function  for  n-1  Is  de¬ 
fined  for  all  and  (xj).  Furthermore,  when  2  <  n  <  N, 

we  see  from  equation  (2)  that  if  rK(dn)  is  defined  for  all  xfX£Sx 
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and  dn€Sd  (xn),  than  tha  return  function  meets  the  monotontclty  re¬ 
quirement  that  ia  defined  In  Chapter  11  for  a  deterministic  decision 
process.  The  previous  statement  Is  true  because  ST,  Su  >  0  and  since 
rK(dn)  1*  a  probability,  then  1  -  rK(dn)  >  0.  Thus,  application  of 
the  basic  functional  equation  for  a  deterministic  decision  process, 
equation  (II-5),  will  yield  optimum  values  of  fn(xn)  for  all  1  <  n  <  N. 

Substituting  equations  (1)  and  (2)  above  along  with  the  appro¬ 
priate  transformations  and  set  definitions,  equations  (II 1-3) ,  (III-4) 
and  (IU-5)  Into  equation  (11-5)  we  get  the  following  recursive  rela¬ 
tions  for  the  simple  duel  with  probability  of  kill  as  an  objective. 

If  n  -  1,  x1€SXi, 

f^xj)  «  Max  [ST  rK(d!)]  -  ST  rfc(Xl)  (3) 

0  £  dl  <  *1 

since  r^(dn)  is  assisted  to  be  monotonlcally  nondecreasing. 

!f  2<n<N,  x^S^, 

W  "  ST  „  Max  {rK(dn)eS,j[l-rK(dn)]fn.1(xr-dn)l(4,) 
Nonrecurslve  Form 

LeC  Rn^dn’dn-l»“"”»dl)  denote  the  probability  of  kill  obtainable 
on  passes  n,—  ,1  for  the  allocation  <<*n»<in.i,— .d^.  With  one  pass 
remaining,  we  have 


R1  (d  1 J  “  ST  Tjj-  (d  j  J 


(5) 


I 

1 


With  two  passes  remaining,  ue  have 


R2  (d2  fdf)  ■  S-ji  rK(d2)  ♦  S*j*  Sy  1  •  rK^2^-5  ^1^1^ 

-  ST  rK(d2)  ♦  sT2  Su  [1  -  rK(d2)]  r^tdp  (6) 


With  three  passes  remaining,  we  have 


®3^3»**2 *  ®T  ®u  R2^2»^l^ 

“  ST  rK^d3^  *  ST^  Su  [1  "  rK^3^J  ^^2^ 

♦  ST3  Su2  [l  -  rj^dj)]  [l  -  rK(d2)]  t^(dj) 


We  can  now  construct  the  expression  for  the  N  stage  return  for  a  given 
allocation,  (dN,dN>>l,  — .dj). 


N  ...  N 

RN(dN,dN_i,— ,dj)  -  ST  £  <SX  Su)  n  [l-ry^)  tyCdjj.^j) 

1«1  n»N-l+2 


where  for  an  arbitrary  function  of  n,  say  g(n). 


tt  g(n)  “  1.0 
n»Nel 


Considering  the  constraints,  equations  ( I II- 1 )  and  (II1-2),  we  can 
state  the  optimization  problem  as  follows. 

f  N  1-1  N  - 

fN(xN)  -  Max  •<  ST  £  (STSU)  n  ^l-r^fd,,)] 

$<*1  L  1*1  n«»N-le2 
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subject  to 

N 

£■  djj  <  (H) 

n- 1 

dn>0  ;  l<n<H  (12) 

Wi  :lng  the  problem  in  the  form  of  equations  (10),  (11),  and 
(12)  is  perhaps  helpful  in  understanding  the  origin  of  the  recursive 
equations.  It  also  promotes  an  appreciation  of  the  simplicity  of  the 
recursive  solution  method.  The  development  of  nonrecursive  problem 
statements  and  the  derivation  of  recursive  relationships  therefrom 
are  shown  in  Appendix  A  for  some  of  the  more  general  problems  that  are 
treated  herein. 


The  Special  Case  where  T  -1,0 
Equations  (10),  (11),  and  (12)  are  not  very  encouragin'-  ?rom 
the  standpoint  of  classical  optimisation  techniques,  however,  we  can 
gain  one  useful  bit  of  insight  by  studying  the  recursive  relations, 
equations  (3)  and  (4).  We  will  show  that  if  4^  -  1.0  and  other  con¬ 
ditions  are  appropriate,  all  weapons  should  be  delivered  on  the  first 
pass. 

Suppose  the  stage  return  can  be  expressed  as 

rK(dn)  -  1  -  6dn  ;  0  <  6  <  1  (13) 

This  is  the  same  as  equation  (III-1S)  with  -  1.0.  Then  from 
equation  (3),  when  n  »  1, 

fi(xj)  -  ST  (1  -  0*1) 


(14) 


i 


We  will  now  hypothesise  that 


fn-l<*n-l>  -  sT  (1  -  O*""1)  -  ST  (1  -  eXn'dn) 

Then  from  equation  (4) 

W  -  ST  Max  {  i-edn  ♦  s  edn  sT(i-eXn‘dn)l 
°£dn<*n  ^  T  J 
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(15) 


(16) 


or 


(17) 


fn<*n>  "  ST  Max  -f  l-0dn  (1-STSU)  -  S ^  0Xn} 

0  <  “„  <  ^  L  ru  i 

Let  *  ^^n^n^*  Then  we  note  that 

Q<Xn,d«  *  1)  -  Q(xn,dn)  -  -  (1  -  STSu)(0  -  l)dn  (l8) 

Since  STSU  <  1.0  and  6  <  1.0,  the  above  difference  Is  non  negative. 

It  follows  that  dn*  .  xn  and  therefore  substituting  in  equation  (17) 


gives 


fr.(xn>  -  ST  Q(*n,Xn)  -  ST  (l-0Xn) 


(19) 


Thus,  when  rK(dn)  is  given  by  equation  (13),  we  have  shown  that  ft(Xl) 
has  the  fora  of  equation  (19)  and  if  ^(x^j)  has  the  form  of  equa¬ 
tion  (19),  then  so  does  fn(x„);  so  by  mathematical  induction,  for  all 
1  -  "  -  N’  dn  “  xn  and  V’Si)  has  the  fora  of  equation  (19), 

The  conclusion  is  that  in  Che  PR  duel,  if  r(dn)  has  the  fora 
of  equation  (13),  then  all  weapons  should  be  delivered  on  the  first 
pass,  i.e.,  take  dN*(xN)  -  xN. 


\ 
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Parametric  Investigation 


The  parametric  Investigation  is  not  as  convenient  for  the  P^ 
duel  as  It  was  for  the  E^  duel  of  Chapter  III.  If  we  assune  r(.(dn) 
can  be  expressed  by  the  analytic  fora,  equation  (III-18),  the  PK  duel 
objective  function,  equation  (10)  becomes 


f  *»  i.i 

fN(x,  )  -f  S,  Max  "S  £  (SfSu) 1 
d  ,---  ,d  v.  i**l 

tt  [l-  y  (i-edn)]a-sdN“l*1)f 
n«N»le2  J 

(20) 

From  equation  (20),  the  optimal  allocation  depends  on  the  three  quan¬ 
tities  V  ,  0,  and  (S-jSu)  for  given  values  of  xN  and  N.  In  the  para¬ 
metric  Investigation  of  the  E^  duel,  the  optimal  allocation  depended 
only  on  9  and  (STSU)  for  given  x^  and  N  and  was  independent  of  the 
value  of  vt/  . 

Figure  3  illustrates  the  fg(dn)  function  for  -  1.0  and  fcr 
the  various  values  of  0  that  are  used  here.  Since  f  Is  a  multipli¬ 
cative  constant,  the  curves  of  Figure  3  can  be  made  to  apply  for  any 
^  by  simply  changing  the  ordir.ifi  scale. 

Tables  7  through  10  show  the  optimal  allocation  versus  (S^y) 
and  e  for  various  values  of  'V  .  Table  10  reflects  the  result  for 
the  special  case  where  V  ■  1.0.  The  entry  in  each  block  in  these 
tables  Is  the  vector  (d^.-.-.d^)  where  zeros  are  omitted.  It  is  clear 
In  the  P^  duel  as  it  was  in  the  Ejj  duel  that  if  n^  >  n2»  then  dn*  >  dn^ 
because  of  the  discounting  structure  and  the  fact  that  the  functional 
form  of  r^(dn)  is  the  same  for  all  n. 


IMUM  WEAPON  ATXOCATION,  (djj 
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We  see  that  with  other  factors  held  constant,  increasing  sur¬ 
vival  probability  calls  for  more  uniform  distribution  of  weapons  among 
Che  passes.  The  same  type  of  trend  occurs  when  0  #r  decreases  in 
value  with  other  factors  held  constant. 

The  difference  between  the  Eg  duel  results  and  the  Pg  duel 
results  can  be  appreciated  by  comparing  Table  10  with  Table  2.  Ignore 
for  the  moment  the  different  interpretation  of  r(dn)  in  the  two  models. 
The  numerical  inputs  are  the  same  for  both  of  these  sets  of  results. 


With  the  same  numerical  input  values,  the  Eg  duel  and  the  P^  duel  can 
have  quite  different  optimal  allocations. 


Generalising  the  PR  Duel 

A  model  for  the  PK  duel  with  probabilistic  acquisition  and  mul¬ 
tiple  modes  of  attack  will  now  be  developed.  This  model  is  similar  to 
the  model  for  the  analogous  Eg  duel  that  was  discussed  in  Chapter  IV. 

The  salvo  effectiveness  function  will  be  r„(D  )  ■  r„(d  ,k  ); 

K  n  K.  n  n 

it  will  represent  the  probability  of  at  least  one  hit  as  a  function  of 
salvo  size  and  mode  of  attack.  The  maximum  n  stage  return  will  be 
fnl(Xn)  ■  fnl^xn*8n^’  lt  represent  the  maximum  probability  of  at 

least  one  hit  when  the  system  is  in  Markov  state  1,  n  passes  remain, 
xn  weapons  remain,  and  the  probability  of  surviving  the  remaining  n 
passes  must  be  at  least  sn. 

The  following  aspects  of  this  model  are  identical  to  the  cor¬ 
responding  aspects  of  the  model  for  the  Eg  duel  with  probabilistic 
acquisition  and  multiple  mcdes  of  attack  that  was  developed  in  Chapter 


IV.  The  Markov  state  definitions  are  the  same  as  those  given  in 
Chapter  IV.  The  Markov  state  transition  probabilities  are  given  in 


mMani'WiHur *  imwiiininiamnsiwawi  aitaiiisiiwiikiiUwmmiWiM  ■*. : 
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figure  6.  The  basic  Crons format  ion  equations  for  the  residual  state 
variables  are  given  by  equations  < IV— 15)  and  (IV-16).  The  basic  sets 
Sx  ,  S  ,  Sd  (x  )  and  Sk  (sn)  are  defined  by  equations  (IV-21)  through 

*n  n  n  n 

(IV-24).  The  required  modification  of  the  transformation  equations  and 
the  set  definitions  to  account  for  the  discrete  nature  of  numerical 
calculation  related  to  sn  are  given  by  equations  (IV-34)  and  (IV-33), 
respectively. 

The  return  functions  for  this  model  have  the  same  form  as  those 

for  the  simple  PK  duel  but  they  must  be  modified  to  account  for  the 

presence  of  multiple  acquisition  states  and  multiple  modes  of  attack. 

When  one  pass  remains,  weapon  delivery  is  associated  with  tran¬ 
sition  to  Harkov  state  3  and  weapons  are  not  delivered  when  transition 
is  to  Markov  states  1,  or  2.  The  probability  of  surviving  to  the  point 
of  weapon  release  and  the  salvo  effectiveness  depend  on  the  mode  of 
attack.  Thus,  If  n  ■  1  and  1  <  1  <  3, 

*n j  Lxi*  Di.  fo,j(*o>]  <21> 

-  0  ;  j  -  1,2 

-  ST(k1)  ^(dpkj)  ;  j  -  3 

When  more  than  one  pass  remains,  the  foregoing  statements  still 
apply.  Furthermore,  we  can  use  the  same  argiment  that  we  used  In  re¬ 
lation  to  the  simple  Pk  duel  to  justify  an  expression  for  the  compo¬ 
sition  of  the  stage  n  return  with  the  maximum  return  for  the  remaining 
n-1  stages. 

If  transition  Is  to  Markov  state  3,  weapon  delivery  is  implied 
and  for  given  and  Dn,  the  probability  of  target  kill  occurring  on 
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pass  n  Is  ST(kn)  r(dn,kn).  The  probability  that  the  aircraft  survives 
pass  n,  fails  to  kill  the  target  on  pass  n,  and  then  kills  the  target 
on  one  of  the  n-1  remaining  passes  Is  given  by 

ST(kn)  Su(kn)  [l  -  rK(dn,kn>]  fn.i,3(xn.1,sn-l) 

If  transition  is  to  Harkov  states  1  or  2,  no  weapons  are  delivered  on 
pass  n  and  the  probability  that  target  kill  occurs  on  pass  n  is  zero. 
The  probability  that  the  aircraft  survives  pass  n  and  kills  the  target 
on  one  of  the  remaining  n-1  passes  is  ST(kn)  Su(kn)  fn-i,  j  (5tn-l»sn-l^ 
where  ^  and  sn.j  are  given  by  the  appropriate  transformation  equa¬ 
tions  .  We  can  sunsnarlze  Che  foregoing  as  follows.  If  2  <  n  <  N  and 
for  all  1  <  1  <  3, 

gnij  LVDn*fn-l,j<Xn-l^  *  ST(kn)Su<kn)fn-l , j (xn-l ,sn-l)  5  J  "  1,2 

-  ST(kn)rK(dn,kn)  ♦  ST(kn)Su(kn)  OrK<dn’kn>]  fn-l,3<*n-l’sn-l>  5  -J"3 

(22) 

If  rK(dn,kn)  Is  defined  for  all  xnCS„  ,  dn€Sd  (xn^* 

n  n 

1  <  n  <  H,  then  these  return  functions  satisfy  the  sufficient  condi¬ 
tions  for  optimality  as  defined  in  Chapter  II.  This  is  trivially  true 
for  equation  (21).  Equation  (22)  has  the  aanotonicity  property  for 
all  n,  I,  J,  3^,  and  Dn  in  their  respective  set9  because  under  all 
conditions 

ST<kn>  Su(kn>  t1  -  rK<dn-kn>]  2.  0  <23> 

Likewise,  the  equivalence  condition  is  satisfied  because  8njj  ['3  Is 
a  linear  function  of  fn_i fj (xn-i*sn-l^' 
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The  various  pares  are  now  available  so  the  Pj.  duel  with  proba- 
bllistlc  acquisition  and  multiple  modes  of  attack  can  be  optimized  by 
using  the  functional  equation  for  a  Markovian  decision  process,  equa¬ 
tion  (11-18).  Substituting  values  gives  the  following.  in  stating 
these  equations,  we  will  ignore  the  practical  problem  that  sn  cannot 
be  treated  as  a  continuous  variable  when  making  numerical  calculations. 
It  will  be  understood  that  the  appropriate  modifications  are  used  when 
calculations  are  made. 

If  n  ■  1  and  for  all  1  <  i  <  3, 

3 

fli(xi,si)  -  Max  £  py^)  ST(kj)  *’K<d1,k1) 

°<dl<X!  J-l 

kl  ^Sk1(sl) 

-  Max  LPi3<ki)  ST(k|)  rK^xl*kpJ  (24) 

kl€  Ski(Sl) 


if  we  assume  that  rk(dj,kj)  is  a  monotonlcally  non  decreasing  function 
of  dj. 

If  2  <  n  <  N  and  for  all  1  <  i  <  3, 


fnl(xn’sn>  *  o  Plj«'n)ST<kn)Su<kn>*n-l,j(>4i. 


“n^  <xn) 


Pl3<kn>  [  ST<kn>rK<dn*kn>  ♦  ST<kn)Su<kn>  0*K<V 


fn-l,3 


(W 


8 


n 


sT^kn^su^kn^ 


(25) 
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Equation  (25)  can  be  simplified  to 


ST(kn)pi3<kn>  n  [rK(dn*kn)*Su<kn)L1’rK(dn»kn)^ 

0  ^  dn  ^  ’‘n 


fn-l‘3(Xn“dn,ST(kn)Su(kn)). 


(26) 


Numerical  Example 

The  application  of  equations  (24)  and  (26)  will  be  illustrated 
by  using  a  problem  that  is  very  nearly  the  same  as  the  example  problem 
of  Chapter  IV.  In  this  example,  N  ■  3  and  xN  -  8.  Acquisition  is 
probabilistic  and  is  characterised  by  the  values  given  in  Table  3  and 
Figure  7.  The  survival  probabilities  are  given  in  Figure  8. 

The  salvo  effectiveness  function,  rR(Dn),  used  for  this  problea 
differs  from  the  one  used  in  the  example  of  Chapter  IV.  First,  its 
Interpretation  Is  different  since  rK(Dn)  is  the  probability  of  target 
kill  versus  salvo  size  and  mode  of  attack.  Second,  the  functional  fora 
of  rK(Dn)  is  the  same  as  before,  i.e.,  equation  (I 11—18)  and  the  values 
of  0  for  the  four  modes  of  attack  are  unchanged,  but  we  now  have 
V  -  0.25.  The  values  of  '■K  and  0  and  the  resulting  r^W^i^)  func¬ 
tions  for  the  four  modes  of  attack  are  shown  in  Figure  12. 

Calculations  were  made  with  xN  •  8  and  s^  -  0.976.  Weapons  are 
assumed  to  be  allocated  in  groups  of  one.  The  survival  constraint  was 


effectiveness  versus 
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varied  in  Increments  of  A  s  •  0.002.  These  values  are  the  sane  as 
they  were  in  the  example  of  Chapter  IV. 

Tables  11,  12,  and  13  show  extracts  from  the  principal  tables 
of  results.  These  tables  respectively  show  maximum  probability  of 
kill,  best  salvo  site,  and  best  mode  of  attack  versus  number  of 
weapons  remaining,  attrition  constraint,  and  Harkov  state.  Note  that 
here  as  before,  the  first  entry  in  each  block  in  Tables  11,  12,  and  13 
applies  when  the  system  is  in  Markov  state  1  (target  acquisition  has 
not  yet  occurred).  The  second  entry  in  each  block  applies  when  the 
system  is  in  Markov  state  2  or  3  (target  acquisition  has  occurred). 

Figure  13  shows  an  illustrative  optimum  attack  policy  corres¬ 
ponding  to  1  •  8j  •  0.012.  This  figure  was  constructed  from  data  in 
the  series  of  tables  of  results  that  Tables  11,  12,  and  13  were  ex¬ 
tracted  from.  The  optimum  attack  policy  of  Figure  13  happens  to  be 
identical  to  the  optimum  attack  policy  of  Figure  10  which  applies  to 
the  example  of  Chapter  IV.  Note,  however,  that  not  all  the  results 
are  the  same  for  the  two  examples  as  can  be  seen  by  comparing  Tables 
11,  12,  and  13  of  this  chapter  with  Tables  4,  5,  and  6  of  Chapter  IV. 

The  retum-versus-attrltlon  function  for  this  example  can  be 
read  from  the  tables  of  results  at  n  -  3,  x-j  •  8,  and  assuming  the 
system  is  In  Markov  state  1  (the  target  has  not  yet  been  acquired). 
Some  of  these  values  appear  in  the  appropriate  positions  in  Table  11. 
Figure  14  shows  the  resulting  return- versus- attrition  function  for 


this  example  problem. 


Attrition  Constraint 
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Nunerlcal  Example:  Reduced  Salvo  Effectiveness 
For  another  example  of  the  application  of  the  PK  duel  model, 
equations  (26)  and  (26),  an  Interesting  outcome  results  If  the  salvo 
effectiveness  is  made  small  while  the  acquisition  and  survival  inputs 
are  maintained  the  same  as  they  were  in  the  previous  example,  l.e. ,  as 
in  Figures  7  and  8.  If  the  saivo  effectiveness  function',  are  as  shown 
in  Figure  IS,  the  optlmim  policy  at  N  ■  3  Is  Illustrated  by  the  ex¬ 
tracts  shown  Tables  16  and  15, 

The  Interesting  feature  Is  that  for  ail  values  of  attrition 
constraint  greater  than  or  equal  to  0.012,  tero  weapons  are  delivered 
on  the  first  pass.  Note  also  that  the  "safest"  mode  of  attack  is 
employed,  l.e.,  k^*  »  6. 

The  complete  attack  policy  for  1  ••  s3  ■  0.018  Is  diagrammed  In 
Figure  16.  This  policy  says  to  make  the  first  pass  using  the  "safest" 
mode  with  no  Intention  of  delivering  weapons.  The  purpose  of  the  first 
pass  Is  to  acquire  the  target.  Once  target  acquisition  has  occurred, 
the  least  conservative  mode  of  attack,  l.e.,  the  most  effective.  Is 
employed  and  all  weapons  are  delivered.  If  the  target  is  not  acquired 
on  the  first  pass,  the  process  Is  repeated  on  the  second  pass. 


Attrition  Constraint  |  Attrition  Constraint 
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tabie 


BEST  SALVO  SIZE  WITH  RffiUCED  SALVO  EFFECTIVENESS:  <13^X3,83) 


Numbs r  of  Wsapons  Renaining:  (X3) 


TABLE  15 

BEST  MODE  OF  ATTACK  WITH  REDUCBJ  SALVO  EFFECTIVENESS: 
^31  ^3,53) 


Number  of  Weapons  Renaining:  1x3) 


CHAPTER  VI 


GENERALIZING  THE  ATTACKER'S  OBJECTIVE 

An  Aspiration  Level  Duel;  The  Simple  "Pc  Dud" 

The  P^  duel  that  was  discussed  In  Chapter  V  Is  a  special  case 
of  the  duel  that  will  be  discussed  next.  It  Is  possible  that  the 
attacking  aircraft  would  wish  to  maximize  the  probability  of  obtaining 
at  least  C  hits.  This  will  be  referred  to  as  the  "P^  duel.”  The  as¬ 
piration  level  Is  C  hits.  The  Pg  duel  Is  a  special  case  of  the  P^.  duel 
where  the  aspiration  level  Is  one  hit. 

First,  consider  a  simple  P  duel  which  does  not  Include  proba- 

w 

billstic  acquisition  or  multiple  modes  of  attack.  Let  h(J{  ;  dn)  be 
the  probability  function  of  the  number  of  hits,  J?  ,  per  salvo  of  size 
dn  where  X  is  an  Integer  such  that  0  <  <  dR. 

The  state  of  the  system  when  preparing  to  make  pass  n  can  be 
described  by  specifying  values  for  two  state  variables.  The  number  of 
weapons  remaining  is  and  since  we  are  not  for  the  moment  including 
probabilistic  target  acquisition,  dR  weapons  are  assisned  to  be  de¬ 
livered  on  pass  n.  The  transformation  of  xn  is  given  by  equation  (1). 
For  all  1  <  n  <  N, 


xn- 1  -  *n  -  dn 


(1) 
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0.0  If  transition  Is  not  to  state  I,  Is  1.0  If  the  system  la  already 
In  state  I,  and  Is  equal  to  tho  probability  of  surviving  to  the  point 
of  weapon  delivery,  S^.,  if  the  transition  Is  to  state  1  from  some 
other  state.  To  summarise,  If  n  •  1, 


Sltj  L*l.dL.fo,j<xo>]  "  °-° 

•  1.0 

m  Sf 


1  <  l  <  I;  1  <  J  <  I 
1  -  I 

1  <  i  <  I;  j  -  I  (5) 


When  more  than  one  pass  remains  to  be  made  and  transition  is  to 
some  state  other  than  I,  the  probability  of  achieving  at  least  C  hits 
In  the  duel  is  the  probability  of  surviving  pass  n  and  achieving  tho 
remaining  hits  In  the  remaining  n-1  passes.  If  the  system  Is  already 
in  state  I,  the  probability  of  achieving  at  least  C  hits  in  the  duel 
is  1.0.  If  transition  Is  to  state  I  from  some  other  state,  the  proba¬ 
bility  of  achieving  at  least  C  hits  in  the  duel  is  S^.  To  summarise, 
If  2  <  n  <  N, 


gnlj  Lxn,dn*^n-l,j  ^xn-l^ 

*  STsufn.  i  •  1  <  i  <  I 

;  1  <  j  <  I 

-  1.0  ;  1  -  I 

-  Sx  ;  ICICI 

;  J  -  i 


(6) 


tfe  have  nov  defined  all  of  the  parts  of  the  basic  functional 
equation  for  the  Markovian  decision  process,  equation  (11-18). 
Substituting  the  expressions  from  equation  (2)  for  the  Pij(dn)  term. 
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substituting  the  expressions  from  equations  (5)  and  (6)  for  the 
gnjj  [•”]  term  and  using  the  sets  defined  by  equations  (3)  and  (4) 
gives  the  following.  If  1  ■  I  and  for  ail  1  <  n  <  N 

fnI(xn)  -  1.0  (7) 

If  n  -  1  and  1  <  1  <  I, 


dl 

fnCxj)  -  Max  ST  £  h(j^dj)  (S) 

°  <  di  <  xx  Jfc-C-iel 

If  2  <  n<  H  and  1  <  1  c  I, 

f  C  dn 

fni<xn>  •  Max  £  h(J‘i;dn)STSufn-l,  j(xn-l)4ST.  J* .  h(-*;dn) 

0<dn<xn[J-l  A-C-Ul 

(9) 

Since  the  return  functions,  equations  (5)  and  (6)  clearly 
satisfy  the  monotonlclty  and  equivalence  conditions  of  Chapter  II, 
recursive  application  of  equations  (7),  (8),  and  (9)  will  yield  the 
maximum  n  stage  return  and  the  optimum  weapon  delivery  policy  for  the 
simple  Pc  duel. 

Note  that  the  optimum  policy  indicates  the  be3t  act  as  a  func¬ 
tion  of  n,  1,  and  Xj,.  To  Implement  this  policy,  the  pilot  must  know 
the  number  of  hits  already  achieved.  The  implications  of  this  will  be 
discussed  later. 


A  Generalized  Pc  Duel 

In  generalizing  the  model  for  the  Pc  duel  to  include  probabilis¬ 
tic  acquisition  and  multiple  modes  of  attack,  let  h(J;Dn)  -  hi^idj,,!^) 


! 

i 
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be  the  probability  function  of  the  number  of  hits,J(,  achieved  on 
pass  n  when  dn  Is  the  salvo  size  and  mode  of  attack  kn  Is  selected. 

The  maximum  n  stage  return,  fniO^)  •  fnl^xn,sn^  ls  the  maxlmuin  proba¬ 
bility  of  achieving  at  least  C  hits  in  the  duel  when  n  passes  are  yet 
to  be  made,  xn  weapons  remain,  the  probability  of  surviving  the  remain¬ 
ing  n  passes  must  be  at  least  ?n,  and  the  system  is  in  Markov  state  i. 

The  Markov  state  must  now  reflect  both  the  number  of  hits  that 
have  already  been  achieved  and  the  acquisition  status  of  the  system. 

One  possible  approach  would  be  the  definition  of  a  two  dimensional 
vector  to  characterize  the  Markov  state  of  e  system.  We  will  take 
the  approach,  however,  of  defining  Markov  states  In  such  a  wav  that  a 
single  dimension  Markov-state  variable  can  reflect  both  the  nunber  of 
hits  already  achieved  and  the  acquisition  status  of  the  system.  The 
functional  equation  (11-18)  can  then  be  used  directly  to  optimize  the 
return.  To  accomplish  this,  the  Markov  states  are  as  defined  In 
Table  16. 

Figure  18  shows  the  transition  prcbabl lltles  for  the  case  where 
C  ■  3.  Using  this  figure  as  a  guide,  we  can  construct  the  transition 
probabilities  for  the  general  case.  If  1  -  1, 

Plj(Un)  -  1  -  F(A0)  ;  J  -  1 

-  P(Aq>  -  P(A0D)  ?  j  -  2 

-  0  J  3  <  j  <  Cel 

-  PCA^}  h( j-C-2 ;  Dn)  ;  Cv2  <  j  <  2Cfl 

-  P(A0D)  £  h(J);Dn) 

i-c 


;  j  -  I  -  2C*2 


(10) 
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TABLE  16 

DEFINITIONS  OF  MARKOV  STATES  IN  THE  GENERALIZED  l?c  DUEL 


State  Variable 


Def inltion 


i  -  1 
1  -  2 


1  -  3 
i  -  4 


A*  «•  Acquisition  has  not  occurred 
AD*;0  hits  »  Acquisition  has  occurred, 

delivery  did  not  occur  cm  the 
most  recent  pass,  and  no  hits 
have  been  scored 

AD*;1  hit  ■  same  as  above  with  1  hit  scored 
AD*;2  hits  -  same  as  above  with  2  hits  scored 


l  -  C  ♦  1 

i  -  C  ♦  2 

1  -  C  ♦  3 
1  -  C  ♦  4 


AD*;C-1  hits  -  same  as  above  with  C-l  hits  scored 
AD;0  hits  -  Acquisition  has  occurred,  delivery 
occurred  on  the  most  recent  pass, 
and  no  hits  have  been  achieved 
AD;1  hit  -  same  as  above  with  1  hit  scored 
AD;2  hits  -  same  as  above  with  2  hits  scored 


i  ■  2C  ♦  1  AD;C-1  hits  »  same  as  above  with  C-l  hits  scored 


i  -  I  -  2C  ♦  2 


At  least  C  hits  have  been  achieved 


<4 


Vi  <N 

r-  •+  *->  v 

a  £ 

•o<x:  ^ 

CM  O 


V© 

I  Q-C 


to  o 

m  .*  Jj  W 

a  n  —  ^ 

-^  <  .c  ^ 

o  Qo 

< 


w 

•*4J 

^  *{<  H-* 


n  v  ^ 

■  a-e 


V) 
•  «4J 

CM  -I'  ** 
I  Q.C 

"*""’  O 


Pig.  18.— Transition  probabilities  for  the  general  Pc  duel  when 


K  2  <  I  <  Ol, 

Pij(Dn)  -  0 

-  1  -  P(A1D) 

-  PUjD)  h(J-C-i,  Dn) 

dn 

-  P(AiD)  £  h<  ;u_) 

X  "C-i  *2 

If  02  <  i  <  201, 

PijCDn)  "  Pl-C.j^ 

If  l  -  I  -  202, 

PlJ<Dn)  "  0 

»  l .  0 
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;  1  <  j  <  i;  i  <  j  <  Ol 

;  j  -  1 

;  oi  <  j  <  2c+i 

;  j  -  I  -  202  (II) 

;  l  <  j  <  2oi  (I?) 

;  !  <  j  <  2ol 
;  J  -  I  -  202  (13) 


The  residual  state  variable  transformation  relations  are  as 
follows.  For  all  l  <  n  <  N  and  1  <  1  <  202, 


•i 

I 

I 

f 

i 

i 

i 


xn-i  "  *n 

-  xn  ‘  dn 

«  .  sn 

n-1  ST(kn)Su(kn) 

In  the  transformation  equation  (15)  we  are  Ignoring  the  problem  that 
in  making  computations,  sn  must  take  discrete  values.  The  above 
transformation  equations  lead  to  the  sets  defined  by  equations  (IV-21), 
(IV-22),  (IV-23) ,  and  (IV-24). 


;  1  <  j  <  Ol 
;  02  <  j  <  202  (14) 

;  all  j  (15) 
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In  defining  the  return  functions,  we  get  some  compensation  for 
the  complexity  of  the  transition  probabilities.  The  return  functions 
are  Identical  to  equations  (5)  and  (6)  for  the  simple  Pq  duel  except 
that  xn  Is  replaced  by  ■  (xn,sn),  dn  Is  replaced  bv  Dn  -  (dn,kn), 

Sf  and  Su  are  now  Sy(kn)  and  Su(kn),  and  1  -  202. 

All  the  parts  of  the  basic  functional  equation  (11-18)  have  now 
been  defined  for  the  Pc  duel  with  acquisition  and  multiple  modes  of 
attack. 


The  Simple  Expected  Damage  Duel: 


The  Simple  "EQ  Duel" 


The  aspiration  level  objective  of  the  Pc  duel  allows  no  utility 
for  any  number  of  hits  less  than  C  and  no  marginal  utility  for  addi¬ 
tional  hits  once  C  hits  have  been  obtained.  This  is  a  tenable  abstrac¬ 
tion  for  some  situations  where  C  »  1 ,  t.e.,  the  PK  duel.  When  C  >  1, 
not  very  many  applications  come  to  mind.  Almost  any  situation  requiring 
three  hits  offers  some  value  for  two  hits.  The  Pg  duel  is  included 
because  It  fits  logically  into  the  pattern  of  duels  that  are  considered 
and  It  makes  a  convenient  way  of  Introducing  the  duel  that  Is  to  be 
considered  next. 


This  leads  to  the  final  generalization  of  the  duel  objective. 

The  duel,  the  duel,  and  the  Pg  duel  are  all  special  cases. 

Suppose  a  utility  or  damage  level  Is  associated  with  the  number  of  hits 
scored.  If  the  Markov  state  variable  1  reflects  the  number  of  hits 


achieved,  then  a  function  U(i)  can  associate  the  damage  level  achieved 


i 


with  the  Markov  state  of  the  system.  The  damage  level  achieved  on  a 
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sortie  Is  a  random  variable.  Maximising  Its  expected  value  Is  a  reason¬ 
able  objective.  This  will  be  referred  to  as  maximizing  expected  damage. 
The  present  case  will  be  called  the  Ep  duel. 

The  functional  equations  for  optimizing  the  simple  ED  duel  can 
be  obtained  by  simple  modifications  of  the  equations  for  the  simple 
p£  duel  that  was  discussed  in  the  first  part  of  this  chapter. 

In  this  development,  fni  (xn)  Is  the  maximum  expected  vaiie  of  the 
marginal  (or  additional)  damage  achievable  in  the  remaining  n  passes 


when  weapons  remain  and  the  system  is  in  Markov  state  1.  The  quan¬ 
tity  h(J|;dn)  is  still  the  probability  function  of  the  number  of  hits, 

.5  ,  per  salvo  of  size  dn  where  0  <  Ji  <  dn. 

The  transformation  relation  for  xn  is  given  by  equation  (1). 

The  Markov  state  variable  i  has  the  same  definition  it  did  In  the  simple 
Pc  duel  (number  of  hits  achieved  •  1  -  1).  The  Markov  state  transition 
probabilities  are  given  by  equation  (2)  and  Figure  17.  The  quantity  C 
is  reinterpreted  as  the  number  of  hits  associated  with  the  maximum 
damage  level,  i.e.,  additional  hits  are  of  no  further  value.  The  sets 


and  S<in(xn)  are  defined  by  equations  (3)  and  (4). 

In  general  terms,  the  composition  of  the  stage  n  return  with 
the  maximum  return  obtainable  from  the  remaining  n-1  stages  Is  the 
discounted  sum  of  the  marginal  damage  achievable  on  pass  n  and  the 
additional  damage  achievable  on  the  remaining  n-1  passes  starting  from 
the  state  that  results  from  pass  n.  More  specifically,  if  n  -  1  and 
1  <  i  <  I  (reference  to  Figure  17  may  be  helpful). 


siij  Lxi*  V  fo,j<v3 

-  sT  j>(j)  -  u(D]  ;  i  <  j  <  i  (16) 
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If  n  -  1  and  l  -  I, 

sliJ  Cxi*  d  1*  fo,J<‘xo^  “  0  »  J  "  I  (17) 

since  damage  level  saturation  has  already  been  reached  before  making 
ttu1  pass. 

If  2  <n<  N  and  1  <  1  <  I, 

8nlj  Lxn*  dn*  fn-l,j(xn-l^ 

-  ST  [U(J)  -  U(i)]  ♦  STSufn.1|j(xn.1)  ;  l<j<I 

-  ST  [U(I)  -  U(l)]  ;  j  -  l  (18) 

If  2  <  n  <  N  and  i  -  I, 

*nlj  On*  dn»  'n-l.j^n-l^  -  0  *  J  *  1  <19> 

Note  that  in  equations  (16),  (17),  (IS),  and  (19),  the  return  functions 
are  not  defined  for  the  cases  where  i  <  1.  This  is  justified  because 
P{j(dn)  -  0  if  J  <  i. 

Ail  of  the  required  parts  of  equation  (11-18)  have  now  been 
defined  so  the  solution  for  the  simple  ED  duel  can  be  obtained. 

The  General  Ep  Duel 

To  include  acquisition  and  multiple  modes  of  attack  in  tha 
Eq  duel,  h(J;  dR)  becomes  h(JJ;  Dn)  and  fn£(xn)  becomes  fn{(X^). 

U(i)  is  the  damage  level  associated  with  Markov  state  i.  The  Markov 
states  are  defined  in  Table  16.  The  Markov  state  transition  proba¬ 
bilities  are  given  by  equations  (10),  (11),  (12)  and  (13).  The  re¬ 
sidual  stats  variable  transformations  are  given  by  equations  (14)  and 


nu 

(15).  Appropriate  sets  are  defined  In  equations  (IV-21),  (1V-22), 
(IV-23)  and  (IV-24). 

The  return  functions  for  this  case  can  be  constructed  as 
follows.  The  quantity  C  must  be  Interpreted  as  the  number  of  hits 
associated  with  the  damage  saturation  level,  l.e,,  achieving  more  than 
C  hits  Is  no  better  than  achieving  C  hits. 

If  one  pass  remains,  the  expected  return  for  given  Xj^,  Dp  1, 
and  j  is  simply  the  probability  of  surviving  to  the  point  of  weapon 
delivery  times  the  difference  In  damage  level  associated  with  the 
Markov  states  1  and  j.  Thus,  If  n  *  1  and  for  all  1  <  1  <  2o2, 

*nj  C*i»  Di»  fo,j(xo>]  -  sT(ki>  CU<J>  -  »o>3  ;  1  u  <  2(*2 

(20) 

When  more  than  one  pass  remains,  the  expected  return  for  given 
X„,  Dn,  1  and  j  Is  the  probability  of  surviving  to  the  point  of  weapon 
release  on  pass  n  times  the  difference  in  damage  level  associated  w^th 
the  Markov  states  1  and  J  plus  the  probability  of  surviving  pass  n 
times  Che  additional  damage  achievable  on  the  remaining  n  -  1  passes. 
Symbolically,  if  2  <  n  <  H  and  for  all  1  <  1  <  202, 

8nlj  Ol»  Di*  ^n- 1 , j 1^1  * 

sT(kn)  [U(J)-U(D]  4  sT(kn)su(kn)fn.l>j(xn:i)  ; 

1  <  j  <  202  (21) 

xhe  case  where  U(j)  <  U(l)  will  occur  among  the  array  of  com¬ 
binations  of  1  and  j  that  are  covered  by  equations  (20)  and  (21). 


r 
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This  case  does  not  create  a  problem  because  the  transition  probabili¬ 
ties,  Pjj(kn),  associated  with  all  such  cases  are  aero.  This  can  bo 
verified  by  oxamlnlng  Figure  18. 

All  of  the  parts  of  equation  <11-18)  have  now  been  defined  so 
the  maximum  expected  damage  and  the  optima  policy  can  be  determined 
for  the  general  duel.  The  optimal  policy  tells  the  pilot  how  many 
weapons  to  deliver  and  what  mode  of  attack  to  adopt  depending  on  the 
number  of  passes  remaining,  the  number  of  weapons  remaining,  the 
acquisition  status,  and  the  number  of  hits  that  have  been  achieved. 

The  existence  of  real  situations  in  which  the  pilot  knows 
exactly  how  many  hits  have  been  scored  is  debatable.  Conversations 
with  experienced  pilots  Indicate  that  the  pilot  generally  does  not 
know  how  many  hits  have  been  scored  but  he  is  not  completely  Ignorant 
of  the  effectiveness  of  his  passes.  He  may  be  able  to  watch  the  effect 
during  pullout  or  a  fellow  pilot  way  make  observations.  Further, 
whether  or  not  the  target  can  be  observed,  the  pilot  has  same  idea  of  ( 

whether  he  has  made  a  good  delivery.  The  ability  of  the  pilot  to  esti-  j 

« 

mate  salvo  effectiveness  is  highly  variable  depending  on  the  condition*  \ 

of  the  attack  and  the  nature  of  the  target.  > 

In  some  cases  it  may  be  appropriate  to  act  as  if  the  number  of  1 

| 

hies  scored  or  current  damage  level  at  each  stage  Is  known.  In  such  * 

cases,  the  previously  discussed  methods  will  yield  optimal  tactics.  | 

j 

Implications  of  Unobservable 
Markov  State  Transitions 

It  is  Interesting  to  see  what  Is  Involved  under  the  assumption  j 

that  the  pilot  has  no  information  at  all  about  the  effectiveness  of  hla 
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previous  passes.  The  simple  Pc  duel  will  serve  as  a  vehicle  for  ex¬ 
amining  chls  matter. 

When  the  Markov  state  transitions  are  unobservable,  the  simple 
Pc  duel  becomes  a  Markov  decision  process  with  unobservable  transi¬ 
tions.  Such  a  process  vas  discussed  in  Chapter  II.  The  probability 
function  for  the  Markov  state  of  the  system,  TTn( 1 ;  dN,— ,dn+j), 
can  be  determined  by  recursive  application  of  the  relation 

I 

nnCJ;TTN»dN* - ♦dn+l^  *  ^  TTnel<l’TrN,dN,'*",dn42)pijtdn)  *22^ 

where  the  Pm (dn)  are  given  by  equation  (2). 

The  functional  equation  for  this  duel  can  be  developed  from  equations 
(8)  and  (9)  for  the  simple  Pc  duel  by  the  same  argument  that  was  used 
to  develop  equation  (11-21)  by  starting  with  equation  (11-18).  We  can 
apply  this  argument  as  follows.  Since  the  Markov  state,  1,  of  the 
system  is  only  known  probabilistically  at  each  stage,  the  maximum  n 
stage  return  is  the  maximum  expected  value  where  the  expectation  is 
taken  over  the  random  variable  1.  Since  this  maximum  n  stage  return 
depends  on  the  Initial  probability  function,  TTjj,  and  it  depends  or.  the 
decisions  DN,—  ,Dwl,  It  Is  denoted  fn(X^,  ttn,  1>n,—  ,Dn4l).  Tho 
return  function  is  Indicated  by  gnlj  [* A*  Dn*  fn-l(xn-l»  TTN»DN*””“ ‘DnX 
and  lu  the  present  application  to  the  simple  Pc  duel,  tne  return  func¬ 
tions  are  similar  to  equations  (5)  and  (6).  If  n  -  1, 

«Hj  L5:l»dl*fo(xo’T,N»dN‘’"“'dl)]  "  °-°  i  *  <  1  <  I;  1  <  j  <  I 

-  1.0  ;  i-I 


ST  ;  1  <  1  <  If  J  -  I  (23) 
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If  2  <n<  N, 
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*ni j  ^xn,dn*^n-l^xn-l,nM,dN»“""*dn^ 

“  sTsuf n- 1  ^xn«  i  »TTN»dN ’ ,dn ^ 

;  1  <  1  <  15  1  <  J  <  1 

-  1.0 

S  1  -  I 

•  Sq. 

;  1  <  1  <  1;  j  -  I  (24) 

Now,  substituting  the  Markov  state  probability  functions  from  equa¬ 
tion  (22),  the  Markov  state  transition  probabilities  from  equation  (2), 
and  the  return  functions  from  equations  (23)  and  (24)  into  equation 
(11-21)  gives  the  following  functional  equations  for  the  Pc  duel  with 
unobservable  Markov  state  transitions. 

If  n  m  1, 


f  1  (xj *TTjv,,d^,,---d2 )  - 

I  dl 

Max  Z  TMl;TTN«dN,--“,d2)ST  £  h(Jt;d,) 

0  "S.  di  <  X1  1-1  A-C-lel 

(25) 


If  2  <  n  <  N 


fn(xn’TTN*dN»"""dn+l)  “ 


Max  Z  TTnO;rTN*dN»"““*dn*l^ 

°<dn<*n  i"1 


r  c 


Z  h(j-i;dn)STSufn.l(xn.l,nN,dN, — ,dn) 
■  J-i 


♦  S„ 


2  h(i 


X  -c-i+i 


(26) 


1 


>r.!i!t,'jrr**,pr'  TlIT*  ^  1 ' 


As  indicated  in  Chapter  II,  the  high  dimens tonality  seems  to  make 
equations  (25)  and  (26)  impractical  to  implement. 

The  prospective  difficulty  of  implementing  equations  (25)  and 
(26)  illustrates  a  far  reaching  difficulty  in  the  study  of  military 
duels  and  in  studies  of  many  other  areas.  The  solution  methods  for 
Markovian  decision  processes  that  are  discussed  in  this  work  and  by 
Howard  (17),  the  solution  methods  for  stochastic  games  discussed  by 
Chames  and  Schroeder  (11)  and  by  Shapley  (23),  and  other  related 
solutions  all  provide  an  optimal  policy  or  strategy  that  indicates  how 
to  act  as  a  function  of  the  state  of  the  system.  This  always  pre- 
supposes  a  perfect  knowledge  of  the  state  of  the  system  on  the  part 
of  the  actor.  The  complications  that  we  have  faced  in  this  section 
are  indicative  of  the  problems  that  arise  when  perfect  knowledge  of 
the  state  of  the  system  can  not  be  assisted. 
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CHAPfER  VII 


MULTfPLE  AIRCRAFT  RAIDS 


General  Considerations 


Discussion  in  previous  chapters  has  involved  the  duel  between 
a  single  aircraft  and  a  defended  target.  This  duel  has  been  charac¬ 
terized  by  the  retum-versus-attritlon  function.  Such  functions  were 
computed  in  previous  chapters  for  examples  of  the  simple  Eg  duel 
(Figure  5),  the  general  Eg  duel  (Figure  11)  and  the  general  PR  duel 
(Figure  14).  A  retum-versus-attrition  function  along  with  the  state¬ 
ment  of  optimal  att<-  jk  policy  at  each  attrition  level  is  informative, 
but  is  generally  not  adequate  for  decision  making.  It  offers  no  indica¬ 
tion  of  which  attrition  level  should  be  adopted.  The  purpose  of  this 
chapter  is  to  shed  some  light  on  the  selection  of  an  attrition  level 
for  the  duel,  i.e.,  selection  of  the  aircraft's  attack  policy. 

The  retum-versus-attritlon  function  represents  a  tradeoff 
between  effectiveness  and  cost.  Our  approach  will  be  to  minimize  the 
cost  of  achieving  a  given  level  of  effectiveness.  In  this  report, 
cost  is  In  terms  of  expected  aircraft  losses.  The  units  of  return 
depend  on  the  situation. 

A  raid  is  visualized  as  follows.  A  group  of  R  aircraft  departs 
from  its  base  and  penetrates  enemy  area  defenses  to  the  vicinity  of 
the  target.  Each  aircraft  attacks  the  target  according  to  a  prede¬ 
termined  policy.  When  the  attack  is  completed,  the  aircraft  penetrate 
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enemy  area  defenses  and  return  to  their  base.  The  t  •  •.  problems  In¬ 
clude  specifying  the  raid  size  R  and  the  attack  p  ...  that  each  air¬ 
craft  will  follow. 


target  where  hits  are  not  all  on  the  same  object  Is  a  dispersed  supply 
depot  consisting  of  many  small  supply  caches  defended  by  a  cormon 
defense  system.  If  the  aircraft  were  to  attack  a  different  supply 
cache  on  each  pass  and  If  r^(dn>  Is  Interpreted  as  the  expected  hits 
per  salvo,  then  would  be  the  maximum  expected  hits  per  duel. 

Note  that  Tjj(dn)  might  also  be  Interpreted  as  the  probability  of  kill¬ 
ing  a  supply  cache  versus  salvo  size  In  which  case  ifl(Xjj)  would  be  the 
maximum  expected  caches  killed  per  duel. 


The  Retum-Versus-Attrl cion  Function 

In  considering  the  retum-versus-attr Ition  function  that  Is 
produced  by  the  models  of  the  previous  chapters,  the  choice  Is  among 
a  number  of  different  attack  policies  which  will  be  indexed  1  <  m  <  M. 
Each  attack  policy  corresponds  to  one  of  the  values  of  the  constrain¬ 
ing  probability  of  the  aircraft  surviving  uhe  duel,  sjj,  where  sN€  Ss  . 


The  return  for  the  generalized  E^  duel,  i ^XN*SN^ *  *s  the  max*mum 
expected  hits  If  we  assume  that  at  the  beginning  of  the  duel,  target 
acquisition  has  not  yet  occurred,  i.e.,  1  •  1.  For  notational  con- 
venience,  we  will  denote  this  return  as  Up(m).  The  attrition  factor 
will  be  symbolized  in  terms  of  the  probability  of  surviving  the  duel. 

Let  Sjj(m)  be  the  probability  of  the  aircraft  surviving  the  duel  when 
policy  m  is  selected. 

Noce  that  SD(m)  is  the  actual  probability  of  surviving  the 
duel  under  the  rotb  attack  policy.  This  may  differ  from  the  corres¬ 
ponding  constraining  value  sN.  Accordingly,  the  first  task  is  to 
modify  the  retum-versus-attrition  function  to  reflect  actual  proba¬ 
bility  of  survival  rather  than  constraining  values.  To  accomplish 
this,  each  of  the  M  attack  policies  Is  evaluated  to  determine  the  re¬ 
sulting  value  of  Sg(m).  It  is  convenient  to  perform  this  evaluation 
by  using  recursive  techniques. 

For  the  generalized  EH  duel,  the  optimal  policy  is  given  by 

^  A* 

dni(xn,sn)  and  kni<xn»sn)»  to  he  abbreviated  dp  and  kp ,  respectively. 

Let  0ni (xn»sn>  be  che  acCual  probability  of  survival  when  n  passes 
remain,  the  system  is  in  Markov  state  i,  xn  weapons  remain,  the  con¬ 
straining  probability  of  survival  is  sn,  and  the  corresponding  optimal 
policy  is  followed.  The  actual  probability  of  survival  can  be  evaluated 
recursively  by  using  the  following  relationships. 


If  n  -  1, 


pll(xl»sl)  "  sT<kl)  SuO<i) 


If  2  <  n  <  N, 


^ni^n-V  “  »Jlj<dn>  sT<kn>  5uO  pn-l,J<xn-l*sn-l>  <2> 
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where  x^^  Is  given  by  crans  format  ion  ^uatlons  (IV-15)  and  sn<>|  Is 
given  by  the  discrete  version  of  the  survival  constraint  transformation 
equation  (IV-34). 

if  the  appropriate  m  is  associated  with  each  s^,  then  for  given 
N  and  x^,  ■  Sjj(m).  Note  that  at  pass  N,  i.e.,  the  first 

pass,  the  system  is  always  assumed  to  be  In  Markov  state  1. 

A  Model  for  Minimising  Expected  Losses 

Suppose  that  the  aircraft  In  the  raid  make  stochastically  inde¬ 
pendent,  statistically  identical  attacks  against  the  target.  Let  SA 
be  the  probability  that  a  given  aircraft  survives  the  area  defenses 
from  its  base  to  the  target  and  suppose  that  the  probability  of  sur¬ 
viving  area  defenses  is  the  same  on  the  return  from  the  target  to  the 
base.  The  raid  site  required  to  realize  Cg  expected  hits  is  given  by 

R(m)  .  - -  (3) 

UflOn)  SA 

where  expresses  the  desired  level  of  accomplishment.  *  The  proba¬ 
bility  that  a  given  aircraft  does  not  survive  the  raid  is  given  by 

1  -  SA2  Sq(o) 

Thus,  the  expected  value  of  the  number  of  aircraft  lost  per  raid  Is 
given  by 

£ 

E  (m)  -  — --S -  (1  -  s2  S0(m) )  (4) 

R  UD<ffl>  SA  A 


*In  making  numerical  calculations,  3(m)  should  take  the  smallest 
Integer  value  no  smaller  than  the  value  of  the  right  side  of  equation 
(3).  This  is  Important  when  that  value  is  small,  especially  If  it  is 
less  than  1.0.  We  have  ignored  this  here. 


f 


The  minimum  expected  value  of  the  number  of  aircraft  lost  per  raid  is 


given  by 


l„.£sJ  hi.  f 

SA  1  <  m  <  M  L  u0(a) 


The  minimizing  value  of  m  will  be  m*  and  R*  ■  R(m*)  will  be  the  opti¬ 
mum  raid  size. 

Key  assumptions  of  this  raid  model  are  as  follows: 
a.  The  same  probability  of  surviving  the  area  defenses  applies 
enroute  from  the  base  to  the  target  and  returning  from  the  target  to 


the  base. 


The  aircraft  in  the  raid  fly  stochastically  independent. 


statistically  Identical  sorties. 

c.  At  least  R*  aircraft  are  available. 

Assumption  a.  could  easily  be  relaxed  but  doing  so  would  only 
add  to  the  complexity  of  this  work  without  adding  substantially  to  its 
content.  To  relax  assionpcion  a.,  we  would  simply  distinguish  between 
the  probability  of  surviving  from  the  base  to  the  target  and  the  proba¬ 
bility  of  surviving  the  return  flight.  The  effect  on  the  equations  in 
the  model  would  be  minor  and  they  could  easily  be  -  -wilf led  to  reflect 
the  change. 

Assumption  b.  is  the  most  Important  one  since  it  is  a  principle 
basis  of  the  raid  model.  That  assumption  Implies  that  all  aircraft 

duels  are  characterized  by  the  same  functions  S»(k  ),  S  (k  ),  r(d  .k 

T  n  u  n  n  n  * 

and  Pjj(kn).  As  was  pointed  out  in  the  first  section  of  Chapter  III, 
the  return  functions  are  indexed  according  to  stage,  n,  which  means 
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that  the  foregoing  functions  could  be  stage  dependent  without  violat¬ 
ing  assumption  b.  as  long  as  each  aircraft's  duel  Is  characterized  by 
the  same  set  of  functions.  This  is  Important  because  the  survival 
probability  functions,Sj(kn)  and  generally  do  depend  on  n. 

Conversations  with  combat  pilots  indicate  that  attrition  on  the  first 
pass  is  generally  much  lower  than  attrition  on  later  passes  because  of 
the  surprise  element.  This  dependence  can  be  reflected  by  making  the 
functions  S-j(kn)  and  Su(kn)  depend  on  n  in  the  recursive  calculations. 

It  Is  usetul  to  be  able  to  reflect  this  dependence  without  invalidating 
the  reld  model. 

The  notion  that  all  aircraft  might  make  statistically  Identical 
attacks  is  reinforced  by  the  following.  When  a  group  of  aircraft 
attack  a  target,  the  attack  is  often  arranged  so  all  the  aircraft  make 
their  first  pass  within  a  short  period  of  time,  i.e.,  each  aircraft 
gets  the  advantage  of  surprise  on  its  first  pass.  If  a  second  pass  is 
Intended,  then  It  seems  reasonable  to  assume  that  the  surprise  element 
is  no  longer  present  for  any  of  the  aircraft. 

A  situation  In  which  the  aircraft  attacks  can  be  assumed  to  be 
stochastically  Independent  and  statistically  Identical  arises  In  the 
all-weather  operation  of  newer  weapon  systems.  Since  these  aircraft 
are  designed  to  make  attacks  under  bad  weather  conditions,  coordina¬ 
tion  of  the  attack  is  difficult  and  therefore  in  designing  weapon 
systems  and  planning  attacks,  it  might  be  assumed  that  when  multiple 
aircraft  are  involved  In  a  raid,  they  operate  independently. 

Assumption  c.  is  implied  by  the  fact  that  raid  size  Is  selected 
and  the  policy  for  the  duel  is  determined  without  regard  for  the  number 
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of  aircraft  available.  If  the  selected  raid  site,  R*,  exceeds  the 
number  of  aircraft  available,  the  objective  of  the  raid  must  be  relaxed. 
In  a  raid  based  on  the  EH  duel,  this  can  be  accomplished  by  reducing 
the  value  of  C^. 

Numerical  Example 

The  example  of  Chapter  IV  Illustrates  the  application  of  the 
model  for  the  EH  duel.  Tables  5  and  6  Illustrate  the  dn^(xn*8n)  *nd 
kn*(xn,sn),  respectively,  that  resulted  In  that  example  and  Figure  11 
shows  the  retum-versus-attrltlon  function  with  the  attrition  In  terms 
of  attrition  constraint.  Figure  19  shows  a  duplicate  of  the  retum- 
versus-attrltlon  function  from  Figure  11  along  with  the  modified  func¬ 
tion  that  results  when  equations  (1)  and  (2)  are  used  to  determine  the 
actual  attrition  associated  with  each  point. 

Suppose  a  raid  Is  to  be  planned  so  as  to  minimize  the  expected 
losses  incurred  In  realizing  an  expected  value  of  the  number  of  hits, 
Cr,  equal  to  10.0.  Suppose  SA  -  0.995.  Applying  the  raid  model  of 
equation  (5)  to  the  Uq(o)  versus  [l  -  SD(m)l  function  shown  in  Figure 
19,  the  expected  value  of  the  number  of  aircraft  lost  is  minimized  by 
choosing  the  minimum  point  on  the  ER(m)  function  shown  In  Figure  20. 

The  minimum  point  occurs  at  the  expected  attrition  level  1  -  SD(m)  - 
.0083.  This  corresponds  to  a  constraining  expected  attrition  value 
of  1  -  sN  »  0.012.  The  optimum  attack  policy  for  this  expected  attri¬ 
tion  level  is  diagrammed  in  Figure  10  and  was  discussed  in  Chapter  IV. 
The  optimal  raid  size  given  by  equation  (3)  for  this  attrition  level 
Is  11.8  aircraft.  The  expected  losses  per  raid  Is  LR  - 


0.215  aircraft. 
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Using  the  Pk  Duel  as  a  Basis 


The  Retum-Versus-Attrltlon  Function 

A  raid  model  based  on  the  Pg  duel  can  also  be  developed.  In 
the  PR  duel  the  quantity  ^fjl^xN’sN^  represents  the  maximum  probability 
of  achieving  at  least  one  hit  g.'.V'wi  that  the  aircraft  survives  the 
area  defenses  from  the  base  to  the  target.  That  probability  will  be 
represented  here  by  KQ(m).  The  modified  retum-versus-attrltlon  func¬ 
tion  for  this  situation  relates  KQ(m)  to  the  probability  of  not  sur¬ 
viving  the  duel,  1  -  Sq(ih). 

A  Model  for  Minimizing  Expected  Losses 

Suppose  raid  size,  R*,  and  attack  policy,  m*,  are  tc  be 
selected  so  as  to  minimize  the  expected  losses  Incurred  In  achieving 
a  probability  KR  of  getting  at  least  one  hit  during  the  raid. 

If  R(m)  aircraft  make  stochastically  Independent,  statistically 
identical  attacks  using  policy  m,  the  probability  that  none  of  the 
aircraft  gets  a  hit  Is 

(1  -  SA  KD(m))R(nl)  -  1  -  Kr  (6) 

2 

The  raid  size  required  to  realize  KR  Is  therefore 

ln(l  -  Kr) 

R(m)  -  -  (7) 

ln(l  -  SA  K0(m)) 


In  making  numerical  calculations,  R(m)  should  take  the  small¬ 
est  Integer  value  wo  smaller  than  the  value  of  the  right  side  of 
equation  (7).  This  is  Important  when  that  value  Is  small,  especially 
if  It  Is  less  than  1.0.  We  have  Ignored  this  here. 


t 
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Since  the  probability  of  a  given  aircraft  not  surviving  Is  given  by 


1  -  S/  SD(m> 


the  expected  value  of  the  number  of  aircraft  lost  per  raid  Is 


ER(m)  -  [I  -  S2  SD(m)]  R(m) 


[1  -  SA2  SD(m)] 


ln(l  -  Kr) 
in(l  -  SA  KD(trt)) 


(8) 


Thus,  the  minimum  expected  losses  per  raid  Is 


L_  -  ln(l  -  K„)  <  Min 
R  R  1  1  <  m  <  M 


1  -  SA2  SD(m) 

L  1  -  SA  ^(m) 


(9) 


It  might  be  assumed  that  if  the  kill  Is  not  achieved  on  one 
raid,  another  raid  vlll  have  to  be  undertaken  and  that  raids  will  be 
repeated  until  the  Job  Is  finally  done.  Suppose  all  the  raids  are  to 
be  stochastically  identical  and  let  target  kill  occur  on  the  1th  raid. 
Then  the  expected  value  of  the  number  of  aircraft  lost  In  killing  the 
target  Is 


Et(b>)  -  £  ER(m)  1  Kr(1  -  Kr) 


1-1 


En  (ffl) 


-  ER(m)  Kr  2  (1*1)(1-Kd)  - 

i-0  ■'R 

Thus,  the  minimum  expected  losses  to  kill  the  target  is 

"\ 


(10) 


Lj.  «  i- 
^  *r 


Min  Eo(m)  ^  -  h 

><■<»  *  J  K. 


rs 

R 


(11) 


HIM  wv 
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So,  under  these  asstnptlons ,  the  raid  site  and  attack  policy  Chat 
minimizes  expected  losses  per  raid  also  minimizes  expected  losses  in¬ 
curred  in  finally  killing  the  target. 

Key  assumptions  for  this  raid  can  be  suaaarlzed  as  follows: 

a.  The  same  probability  of  surviving  the  area  defenses  ap¬ 
plies  enroute  from  the  base  to  the  target  and  returning  from  the 
target  to  the  base. 

b.  The  aircraft  in  the  raid  make  stochastically  independent, 
statistically  identical  attacks. 

c.  At  least  R*  aircraft  are  available. 

d.  Statistically  Identical  raids  are  repeated  until  the  tar¬ 
get  Is  killed  at  which  time  the  raids  cease.  Thl9  assumption  only 
applies  when  computing  Lj. 

Assumptions  a.  and  b.  are  the  same  as  the  first  two  assuaptiens 
listed  in  the  previous  section  and  the  same  consents  apply.  Regarding 
assumption  c.,  if  the  selected  raid  size  exceeds  the  number  of  air¬ 
craft  available,  the  value  of  Kg  must  be  reduced. 

Numerical  Example 

The  example  of  Chapter  V  illustrates  the  application  of  the 
recursive  equations  for  the  duel  and  results  in  the  retum-versus- 
attrltlon  function  that  Is  given  In  Figure  14.  Figure  21  shows  that 
same  retum-versus-attrltlon  function  along  with  the  corresponding 
modified  function  that  is  obtained  by  using  equations  (1)  and  (2). 

This  modified  return-versus-attrltlon  function  indicates  the  maximum 
probability  ef  killing  the  target  versus  the  actual  probability  of 
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the  aircraft  surviving  the  duel.  Recall  front  Chapter  V  that  this 
function  applies  when  three  passes  and  eight  weapons  are  available 
and  target  acquisition  has  not  yet  occurred. 

Suppose  a  raid  is  to  be  planned  so  as  to  minimize  the  expected 
losses  incurred  in  realizing  a  probability  of  K„  «  0.9  of  getting  at 
least  one  hit.  Suppose  SA  -  0.995.  Application  of  the  raid  model  of 
equation  (9)  to  the  KD(m)  versus  [i  -  sD(m)J  function  plotted  in 
Figure  21  Is  illustrated  in  Figure  22.  The  minimum  loss  point  occurs 
at  the  attrition  level  1  -  SQ(m)  -  0.0083  which  corresponds  to  an 
attrition  constraint  value  of  1  -  sN  •  0.012.  The  optimal  raid  size 
given  by  equation  (7)  for  this  attrition  level  Is  10.3  aircraft.  The 
minimum  expected  losses  per  raid  is  LR  »  0.187.  The  optimum  attack 
policy  for  this  attrition  level  is  diagrammed  in  Figure  13, 

Multiple  Target  Raids 

The  raid  model  to  be  developed  next  visualizes  an  operational 
planner  who  has  a  given  number  of  identical  aircraft  available  to  be 
dispatched  simultaneously  on  air-to-ground  attack  sorties.  He  has 
available  an  array  of  targets  that  are  of  varying  difficulty  and  value. 
The  planner  must  decide  how  many  sorties  to  allocate  to  each  target, 
and  he  must  designate  an  attack  policy  to  be  used  for  each  of  the 
raids.  He  must  deal  with  a  tradeoff  between  total  utility  achieved 
and  expected  losses.  It  is  our  intention  here  to  use  the  individual 
duel  results  as  a  basis  for  studying  this  problem. 

First,  consider  a  generalized  raid  model  that  is  based  on  the 
E^j  duel.  Suppose  there  are  T  targets  available  indexed  t  <•  1 ,--- ,T. 
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To  a  large  extent,  the  symbols  used  here  are  simply  the  symbols  used 
for  the  single  target  raid  model  with  the  subscript  t  added.  Thus, 
uot(mt)  and  SDC(at;),  respectively,  are  the  expected  hits  and  the 
probability  of  aircraft  survival  per  duel  with  target  t  when  the  air* 
craft  uses  attack  policy  mt;  SAt  is  the  probability  of  surviving  the 
area  defenses  one  way  enroute  to  or  returning  front  target  t;  Rt  is 
the  sise  of  the  raid  on  target  t  (number  of  aircraft);  ERt<-t>  is  the 
expected  losses  per  raid  on  target  t  when  policy  mc  is  used. 

The  expected  hits  on  target  t  when  the  aircraft  use  attack 
policy  aec  is 

CRt^nlt^  “  Rt  SAt  uDt^mt^  d2) 

Let  uj^z]  be  an  arbitrary  function 

°  <  Ut  [Z]  <  l-°  (13) 

which  represents  the  utility  of  Z  expected  hits  on  target  t.  Assume 

the  utilities  of  hits  on  various  targets  are  additive,  and  let  Kt  be 

T 

the  relative  importance  of  the  targets  where  >  0  and  £  Ac  -  1. 

t-1 

Then  for  a  given  allocation  (Rj^,— and  set  of  policies  (nij,— -tnLj.), 
the  utility  of  all  raids  is 

_  T  T 

U^,  -  £  \t  Ufc  C^Rt(mt^  “  ^  ^t  Ut  t^t^At^t^t^  (*^) 

t-1  t-1 

We  will  suppose  that  the  planner  decides  to  select  (nij,  —  ,mT)  and 
(Rj,---,Rj)  so  as  to  maximise  the  total  utility  subject  to  constraints 
on  the  total  expected  losses,  ERT ,  and  the  total  number  of  aircraft,  R^. 


\ 
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This  can  be  stated  symbolically  as 

T 

Maximize  £  Uc  £rc  SAt  uDt(mt.)3  (15) 

t-1 

subject  to 

T 

£  Rc  <  Rt  (16) 

t-1 

T  T 

£  ^Rt^mt^  "  £l  "  ^At  ^Dt^mt^  —  ^RT  (11) 

t-i  t-1 

Rt  >  0  ;  1  <  t  <  T  (18) 

The  solution  to  this  problem  might  be  used  to  present  the  operational 
planner  with  a  plot  of  UT  versus  E^.  Corresponding  to  each  point  on 
this  curve  Is  an  optimal  set  of  attack  policies,  (mj  )  and  an 

optimal  aircraft  allocation  (Rj  ,RT*).  It  would  be  up  to  the 
operational  planner  to  decide  which  point  on  the  curve  constitutes  the 
most  desirable  operating  point. 

Note  that  there  is  an  alternative  to  the  foregoing  procedure. 

It  might  also  seem  reasonable  to  minimize  the  total  expected  losses, 
Ert,  subject  to  a  constraint  on  the  number  of  aircraft  available,  R^, 
and  a  requirement  on  the  total  utility.  This  procedure  would  not 
necesssrlly  utilize  Rj.  aircraft  if  the  required  total  utility  is  set 
"low”  and  there  may  be  no  feasible  solution  if  the  required  total 
utility  is  set  "high."  For  these  reasons,  the  previous  procedure  is 
selected  for  development. 
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The  problem  seated  In  equations  (15)  through  (18)  can  be  solved 
by  recursive  analysis.  It  will  be  treated  as  a  T  stage  problem  with 
stages  1  <  t  <  T.  The  symbol  Rt  will  represent  the  number  of  aircraft 
allotted  to  targets  1 ,  t.  The  symbol  E^t  will  represent  l  e  ex¬ 
pected  losses  In  attacking  targets  1,— -,t.  The  s'  vector  will  be 
Xfc  «»  (Rt,  Ejjt)  and  the  decision  vector  will  be  Dt  nt).  The 

function  ft(Xt)  will  be  the  maximum  utility  achievab.  i  attacks  on 
targets  1 , t  when  Xt  is  the  state  of  the  system  at  stage  t. 

The  transformation  equations  are 


Rt-1  “  Rt  ‘  8t 


ER,t-l  “  ERt  ‘  ERt 


(19) 


(20) 


Appropriate  sets  can  be  defined 


^  Rt!  0  £  Rt  ^  Rt 
“[  ERt:  0  - 

Eo*. 


ERt  -  ERT  ' 


Smc*ERt»M  " 


mt:  Rc  (.I  -  S 


At  3Dt 


<mt>3  <  ERt 


(21) 


(22) 


(23) 


(24) 


where  Rt,  Rt,  and  m,.  are  Integers,  ERt  takes  only  selected  discrete 

m 

values,  and  E  Is  an  arbitrary  limiting  value.  Note  that  S„  (R. ) 
depends  on  Rt  and  S  (ERT#Rt)  depends  on  ERt  and  Rt.  Equation  (24) 


L 


x 


1A7 


says  chat  a  mode  of  attack  for  target  t  is  chosen  from  among  all  modes 
for  which  the  expected  losses  in  attacking  target  t  is  not  greater 
than  the  expected  losses  allowed  for  attacking  all  of  the  targets 
l  »--•  >  t. 

The  return  functions  are 


g|(X^,0^)  •  a.^  Uj  LRi  sa1  uDl^ml^ 


t  -  1  (25) 


and 

gt  LXt'Dt*ft-l(Xt-l^  “  ^t  ^t  C^t^At^t^t^  *  ^t-l^Xt-l^  ’ 


2  <  t  <  T  (26) 


So  the  functional  equations  can  now  be  written 


fl(X1)  -  Max  4 

RlCS^^) 


Ki  Vt  [RL  SAl  UDl^ral)H 


}  ■ 


"1*  s«1<brpri) 


t  -  1 
(27) 


If  we  make  the  assumption  that  Ut  [z]  Is  a  non  decreasing  function  of 
Z ,  then 


f  i  (X  i )  ■  X 
€  S, 


L  Max  _  -j  Ui  [Ifj.  SAl  ujj]^!)]  > 


;  t  -  l 

(28) 


and 


ft(Xt)  “  Max  _  -i  Xt  Ut  [Rt  SAt  i^t(mc)3  ♦  ^t-l^Xt-lT 

Rt«SRt(Rt)  L  J 

m  e  Z  {Z  o  \  5  2  <  t  <  T  (29) 

®t€  smt'ERt*Rt) 


Equations  (28)  and  (29)  can  be  applied  recursively  to  solve  the  prob¬ 
lem  stated  in  equations  (15)  through  (18). 


A  multiple  target  model  based  on  the  duel  can  be  developed 

in  a  manner  similar  to  the  foregoing  except  that  where  the  foregoing 
deals  with  expected  hits,  we  now  are  interested  in  probability  of 
kill.  Thus,  analogous  to  equation  (12),  we  now  express  the  proba¬ 
bility  of  killing  target  t  with  a  raid  of  Rt  aircraft  each  using 
policy  n>t# 

KRt<“t>  *  C1  -  SAt  <30) 

The  quantity  Kg  t  (nig)  ls  probability  of  killing  target  t  per  duel 
when  policy  mt  is  used.  Where  the  utility  function  Ut  LcRt^mt^3  or 
Ut  [Rt  SAt  appear  in  equations  (14),  (15),  and  (25)  through 

(29),  It  Is  replaced  by  u£  [KRt(mt)]. 

It  would  be  a  simple  extension  of  these  models  to  develop  a 
mixed  generalised  raid  model  where  sane  of  the  target  attacks  would 
be  describable  as  E^  duels  while  other  target  attacks  would  be 
descrlbable  as  PR  duels.  It  would  be  necessary  only  to  use  the 
appropriate  Ut  £  *  ]  or  [  •  ]  for  each  target  in  equations  (28) 
and  (29). 

Special  constraints  such  as  minimum  required  Rt  or  SDt(mt)  for 
various  targets  can  easily  be  included  by  simply  revising  the  sets 
appropriately. 

Multiple  Aircraft  Raid  Model  Based  on  the  Duel 

A  reasonable  extension  of  the  work  that  is  discussed  in  this 
chapter  would  Involve  using  the  Ep  duel  that  was  developed  in  Chapter 
VI  as  a  basis  for  a  multiple  aircraft,  single  target  raid  model. 
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Such  a  model  Is  conceivable  but  it  would  be  considerably  more  complex 
than  the  raid  models  that  are  discussed  In  this  chapter. 

In  the  E^  duel,  the  return  Is  in  toms  of  the  expected  value  of 
the  number  of  hits  which  implies  that  the  aircraft's  utility  for  the 
duel  Is  linearly  proportional  to  the  number  of  hits.  Accordingly,  the 
optimum  policy  for  a  given  duel  is  that  policy  which  maximizes  the 
expected  hits  subject  to  suitable  constraints.  Ctace  having  determined 
the  optimum  policy  for  a  single  duel,  we  can  make  the  assumption  that 
all  aircraft  make  stochastically  independent,  statistically  Identical 
attacks  so  that  the  return  from  the  raid  Is  directly  proportioned  to 
the  raid  size.  Thus,  we  have  a  simple  way  of  determining  the  raid 
size  required  for  a  given  level  of  return.  The  foregoing  Is  the  basis 
of  the  duel  raid  model. 

In  the  PK  duel,  the  return  Is  the  probability  of  at  least  one 
hit.  The  optimum  policy  for  each  aircraft  in  a  raid  is  the  policy 
that  maximizes  the  aircraft's  probability  of  getting  at  least  one  hit 
subject  to  suitable  constraints.  Chce  this  policy  is  determined,  we 
can  find  a  simple  relationship  between  return  and  raid  size,  R.  If 
all  of  the  aircraft  make  stochastically  Independent,  statistically 
Identical  attacks,  then  the  probability  of  the  target  not  being  hit  at 
least  once  is  the  quantity  one  minus  the  probability  of  at  least  one 
hit  per  aircraft  raised  to  the  power  R.  This  provides  a  simple  means 
of  determining  the  number  of  aircraft  required  to  achieve  a  given  re¬ 
turn  and  thus  we  have  the  basis  of  tho  PR  duel  raid  model. 

No  such  simple  relationships  as  those  discussed  above  seem  to 
exist  for  the  ED  duel.  Since  the  return  may  be  a  nonlinear  function 
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of  the  number  of  hits.  It  Is  not  generally  appropriate  to  maximize 
the  expected  value  of  the  number  of  hits  for  an  Eq  duel.  The  opti¬ 
mum  policy  must  reflect  the  nonlinear  character  of  the  utility  as  a 
function  of  the  number  of  hits.  Because  of  the  nonlinearity  of  the 
utility  function,  the  return  associated  with  hits  made  by  a  given 
aircraft  depends  on  the  number  of  hits  made  by  the  other  aircraft  in 
the  raid. 

To  see  this,  suppose  two  aircraft  are  to  make  successive  at¬ 
tacks  on  a  target  such  that  the  first  aircraft  completes  all  of  its 
passes  before  the  second  aircraft  begins  Its  attack.  The  character 
of  the  utility  function  that  governs  the  second  aircraft's  attack 
depends  on  the  outcome  of  the  first  aircraft's  attack,  i.e.,  the  point 
at  which  the  second  aircraft  "enters"  the  overall  utllitv  function 
depends  on  the  number  of  hits  achieved  by  the  first  aircraft.  This 
effect  Is  still  present  but  in  a  more  complex  way  when  the  two  aircraft 
alternate  in  making  their  passes  and  also  when  there  are  more  than  two 
aircraft.  Because  of  this  Inherent  Interaction  among  the  E^  duels, 
there  Is  apparently  no  rigorous  way  to  develop  the  optimum  policy  for 
a  single  Eq  duel  and  then  use  this  policy  In  dealing  with  multiple 
aircraft  raids. 


CHAPTER  VIII 


USER  UNCERTAINTY 

General 

The  discussion  In  previous  chapters  and  In  much  of  operations 
research  centers  Is  on  finding  the  optimum.  Since  we  have  now  de¬ 
veloped  maximizing  techniques  for  at  least  some  air-to-ground  attack 
situations,  some  degree  of  satisfaction  should  have  been  attained. 
There  is,  Indeed,  some  satisfaction  in  contemplating  a  maximizing 
solution  but  as  Is  often  the  case,  overcoming  one  obstacle  only  re¬ 
veals  the  greater  challenges  that  lie  beyond. 

The  techniques  that  have  been  discussed  lead  to  solutions  that 
are  valid  for  a  specific  set  of  input  values.  Figure  23  summarises 
these  inputs  as  they  apply  to  the  single  target  raid  models  and  indi¬ 
cates  the  outputs  that  result. 

In  making  actual  decisions,  there  is  Invariably  some  degree  of 
uncertainty  associated  with  the  values  of  Input  parameters.  A  solu¬ 
tion  that  applies  for  only  one  set  ol  Input  values  may  be  useful  as  a 
reference  for  judging  the  effect  of  other  Input  values  or  might  be 
applied  directly  If  one  Is  willing  to  Ignore  uncertainty.  In  general, 
however,  a  systematic,  quantitative  approach  Is  desirable  to  make  the 
most  rational  decision  based  on  the  best  available  information. 
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The  Effects  of  Uncertainty 

The  approach  to  be  followed  In  dealing  with  uncertainty  Is  de¬ 
signed  to  answer  the  question;  What  will  happen  If  the  Input  param¬ 
eters  take  values  other  than  the  ones  upon  which  the  optimisation  is 
based  ? 

To  answer  this  question  It  Is  first  necessary  to  characterise 
the  available  information  about  the  values  that  the  input  parameters 
might  take.  The  specification  of  intervals  of  possible  parameter 
variation  is  one  mtans  of  reflecting  the  available  Information  about 
these  parameters.  No  indication  is  to  be  included  as  to  how  the  input 
parameter  values  might  varv  within  their  respective  intervals.  Accord¬ 
ingly,  tnls  will  be  referred  to  as  constrained  uncertainty. 

Let  W  be  a  vector  whose  components  consist  of  the  system  inputs. 
Let  the  roost  optimistic  values  and  the  most  pessimistic  values  of  the 
Input  parameters  be  represented  by  the  vectors  W0  and  Wp,  respectivelv. 
Now  define  o  such  that 

w  -  (l  -  a)  wp  T  a  w0  ;  o  <  a  <  l  <l) 

Thus,  W  is  a  convex  combination  of  the  vectors  Wp  and  W0.  As  the  value 
of  a  varies  from  0  to  1,  all  of  the  input  parameters  varv  in  unison 
from  their  most  pessimistic  value  to  their  roost  optimistic  value. 

It  is  also  possible  to  apply  the  foregoing  technique  to  indi¬ 
vidual  input  parameters.  Suppose  the  components  of  W,  WQ,  Wp  are 
wj,  w0j,  wpj ,  respectively,  where  1  <  j  <  J.  Then  a  set  of  aj  values 
might  be  defined  such  that 
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wj  -  <1  -  aj)  wpj  ♦  ajwoj:  0  ^  aj  1  1  »  1  -  J  -  J  (2) 

When  aj  -  a.2  “  —  -  ■  OLj,  then  equation  (2)  Is  equivalent  to  equation 
(1).  As  the  Individual  Cij*s  are  varied  Independent! v  over  their  pos¬ 
sible  combinations  of  values,  the  corresponding  input  parameters  varv 
over  their  possible  combinations  of  values. 

This  method  offers  the  following  features  which  combine  to  pro¬ 
vide  a  systematic  practical  wav  of  studying  the  implications  of  un¬ 
certainty; 

1.  It  tends  to  place  all  Input  variables  on  a  conrion  scale 
with  respect  to  their  range  of  uncertainty. 

2.  It  polarizes  the  Inputs  with  respect  to  their  optimistic 
and  pessimistic  directions  of  variation,  i.e.,  for  anv  input  parameter, 
Increasing  the  corresponding  cij  results  in  the  parameter  taking  a  more 
optimistic  value. 

3.  It  allows  expression  of  basic  inputs  in  non  probabilistic 

terms. 

If  the  method  is  to  be  practical,  the  results  must  be  compre- 
hendable  to  the  decision  maker.  This  consideration  provides  a  strong 
argument  for  emphasizing  the  use  of  equation  (1),  i.e.,  varvlng  the 
input  parameter  values  in  unison.  By  doing  this,  the  extremes  of 
system  performance  are  Included,  some  Indication  of  performance  at 
non-extreme  input  values  is  obtained,  and  the  results  can  be  expressed 
in  relatively  simple  form.  Accordingly,  the  balance  of  the  discussion 
Is  concerned  with  the  use  of  equation  (1), 
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A  Basis  for  Tactics  Selection 
Suppose  that  having  specified  wQ  and  Wp,  the  tactic  that 
optimizes  system  performance  is  chosen  based  on  the  input  parameter 
va lues 

M*  -  (1  -  a')  wp  ♦  a*  UQ  (3) 


It  would  be  possible  to  determine  a  limiting  envelope  of  system  per¬ 
formance  by  letting  a'  vary  from  0  to  1.0  and  optimizing  system  per¬ 
formance  for  each  different  value  of  a*.  As  the  value  of  a'  varies, 
the  optimizing  tactic  changes.  Unfortunately,  only  one  tactic  can  be 
used  in  a  given  situation.  Of  all  the  tactics  that  are  forthcoming, 
as  a*  varies  from  0  to  1.0,  there  is  presumably  one  that  is  at  least 
as  desirable  as  any  other.  Corresponding  to  this  tactic  Is  at  least 
one  value  of  a.'  .  Actually,  there  is  generally  a  range  of  values  of 
a*  corresponding  to  each  tactic  because  a'  is  a  continuous  variable 
and  changes  of  optimum  tactics  occur  in  a  discrete  manner  provided 
only  pure  tactics  are  allowed.* 

To  judge  the  desirability  of  the  tactic  corresponding  to  a 
given  value  of  a',  we  will  examine  what  would  happen  if  the  input 
parameter  values  that  are  actually  realized  differ  from  the  values 
W'  corresponding  to  a’.  The  mechanism  for  accomplishing  this  is  to 
associate  the  realized  values  of  input  parameters  with  the  control 
parameter  a  according  to  equation  (1). 


Mixed  tactics  are  conceivable  here  in  the  same  sense  that 
mixed  strategies  occur  in  game  theory.  Only  pure  tactics  are 
cons idered . 
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Let  F(a;a')  represent  a  measure  of  system  performance  as  a 
function  of  the  realized  input  values  a  when  tactics  optimization  is 
based  on  the  nominal  input  values  corresponding  to  a'.  A  plot  of 
F (a ;a * )  versus  a  for  given  a'  might  be  considered  as  a  profile  of  the 
performance  that  results  from  adopting  the  tactic  corresponding  to  a*. 

To  Illustrate  the  application  of  the  foregoing  technique, 
suppose  F(a;a')  represents  the  cost  of  doing  a  fixed  job  in  a  given 
situation.  The  analysis  might  result  In  the  performance  profiles  that 
appear  in  Figure  24  for  three  different  values  of  a*.  If  a*  -  0.1, 
curve  a  represents  the  profile  of  system  performance  as  the  input 
parameters  vary  in  unison  through  their  range.  Curves  b  and  c  give 
similar  information  for  a'  -  0.5  and  a"  -  0.9,  respectively.  This 
display  presents  to  the  decision  maker  a  highly  digested  summary  of 
the  implications  of  uncertainty  and  his  options  to  control  the  outcome. 

We  can  analyze  Figure  24  in  terms  of  the  principles  of  choice 
undur  uncertainty  (19).  The  minimax  principle  leads  to  selecting  the 
a*  that  minimizes  the  cost  when  a  -  0,  Assuming  that  all  of  the  a 
values  are  equally  likely  and  selecting  the  a'  that  gives  minimum 
expected  cost  is  an  application  of  the  principle  of  insufficient 
reason.  Minimizing  the  maximum  difference  between  the  selected  curve 
and  the  limiting  performance  envelope,  F(a;a)  is  an  application  of 
Savage's  principle  of  mlnlmax  regret.  Finally,  the  display  itself  Is 
in  keeping  with  the  Hurvicz  pessimism-optimism  principle. 

In  the  air-to-ground  attack  problem  and  undoubtedly  In  many 
other  problems,  the  system  performance,  F(a;a')  is  not  a  scalar  quan¬ 
tity  but  must  be  considered  as  a  vector.  For  a  given  a',  botn  return 
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and  cost  vary  as  a  varies.  This  is  unfortunate  because  it  complicates 
the  task  of  the  decision  maker  in  assimilating  the  results,  but  it  in 
no  way  changes  the  basic  idea.  Techniques  for  displaying  and  Inter¬ 
preting  results  when  P(a;a')  is  two  dimensional  will  be  discussed  in 
relation  to  the  numerical  examples. 
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Fig.  24. --Illustrative  System  Performance  Profiles. 
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Treating  Uncertainty  in  the  E^  Duel  Raid  Model 

Consider  now  the  duel  of  Chapter  IV  and  the  corresponding 
raid  model  of  Chapter  VII.  The  system  performance  which  has  been 
represented  by  F(a;a')  is  characterized  by  two  output  quantities. 

Let  Ljj(a;a')  be  the  expected  losses  per  raid  and  let  CR(a;a.')  be  the 
expected  hits  per  raid.  These  two  functions  must  be  evaluated  for 
various  values  of  a'.  In  this  discussion  a  is  associated  with  realized 
input  values  and  a*  is  associated  with  nominal  input  values  where  the 
tactics  optimization  is  based  on  the  nominal  input  values. 

The  number  of  aircraft  per  raid  is  given  by  equation  (VI 1-3) 
which  becomes 

R(a')  -  - -  (4) 

uu(a’;a')  SA(a’) 

where  Upfaia')  is  expected  hits  per  duel  as  a  function  of  a  for  given 
a'»  SA(a)  is  the  probability  of  surviving  area  defenses  one  way  as  a 
function  of  a.  Equation  (4)  simply  makes  explicit  the  fact  that  the 
raid  size  is  determined  entirely  from  nominal  values  and  is  indepen¬ 
dent  of  a. 

The  expected  losses  realized  per  raid  is  given  by 

-  R(a* )  [l  -  sA2 (a)  sD(a;a')]  (5) 

where  Spfaia')  is  the  probability  of  the  aircraft  surviving  the  duel 
as  a  function  of  a  for  given  a'.  The  expected  hits  realized  per  raid 
Is  giver  by 


cR(a;a*)  -  R(a')  sA(a)  uD(a;a*) 


(6) 
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In  evaluating  equations  (5)  and  (6)  as  a  function  of  a,  the 
quantity  SA(a)  is  an  Independent  input  parameter  whose  value  ranges 
from  a  pessimistic  limit  to  an  optimistic  limit  In  accordance  with 
the  value  of  a.  All  other  input  parameter  values  are  also  controlled 
by  a  and  their  effect  is  reflected  by  the  values  of  u^dM')  and 
SD(a;a'),  the  expected  hits  per  duel  and  the  aircraft  survival  proba¬ 
bility  per  duel,  respectively. 

The  functional  equations  (IV-27)  and  (IV-29)  for  the  EH  duel 
can  be  adapted  to  evaluate  uD(a;a').  Let  d^j(xn,sn;a* )  and 

f 

kni (xn,sn;a' )  be  the  maximizing  attack  policy  associated  with  a*.  In 
the  following  equations,  these  will  be  abbreviated  d^  and  k^  respec¬ 
tively.  From  equation  (IV-27),  if  n  ■  1  and  1  <  1  <  3, 

fli<xi,si,a;a')  -  Pi3(kp  a)  sT(kJ,  a)  rH(xltkj,  a)  (7) 

and  from  equation  (1V-29),  if  2  <  n  <  N  and  1  <  i  <  3, 

2 

^ni(xn’sn»a;a,)  "  sT(kn’a)  Su(kn'a)  2  pi1(kn»a) 

j-1 

*  fn-l,j(Vsn-l’  a;a’>  4  ST(kn’  a)  p13<kA’  a)  <8) 

•  LrH(dn*  kA-  a)  4  su(kA’  a)  fn-I.S^n^n-l-aW)] 

where  PjjO«n»a),  sx^kn*  a^»  rH^xn,kn*  a^»  and  su^kn’  correspond  to 

2 

Pij(kn),  ST(kn),  rH(xn,kn),.«nd  Su(kn)  except  that  they  depend  on  a. 

^The  u/\"  is  used  to  distinguish  the  function  "f^j (xn,sn,a;a* ) 
from  the  function  fnj(xn,sn)  that  has  appeared  previously.  Note  that 
these  functions  are  identical  only  if  a  -  a'. 
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Since  the  duel  starts  with  n  -  N,  1  -  1,  xn  -  xN,  and  sn  -  sN, 
the  expected  hits  per  duel  as  a  function  of  a  for  given  a*  Is  given 
by 

UD<a; a*)  -  'fju<xw,  sN,  a;a')  (9) 

In  all  calculations,  sn_|  is  related  to  sn  by  the  transforma* 
tlon  relation  (IV-34)  which  accounts  for  the  discrete  nature  of  the 
calculations.  Note  that  the  survival  probabilities  to  be  used  in 
equation  (IV-34)  are  S^(k^,  a')  and  Su(k^,  a*)-  This  Is  because  when 
the  pilot  carries  out  a  policy,  his  actions  are  based  on  his  nominal 
survival  probabilities,  not  on  the  actual  survival  probabilities  since 
he  doesn't  know  the  actual  values. 

To  evaluate  SD(a;a')  we  use  the  same  method  that  was  used  in 
Chapter  VII  to  evaluate  the  actual  survival  probability.  From  equa¬ 
tions  (VII-1)  and  (VI1-2) ,  If  n  -  1, 

sj,  a; a')  »  sT(k|,  a)  Su(kj,  a)  (10) 

and  if  2  <  n  <  N 

®nl(xn»  sn»  a;a'J  "  (ll) 

3 

^  ST^kn,a^  su^kn»a^  ^n-l,j^xn-l»sn-l»a;CL' ) 

j-1  J 

Finally, 

SpCaia*  >  ->*i  (j^,  sN,  a;a')  (12) 
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Treating  Uncertainty  in  the  Pg  Duel  Raid  Model 


Turning  now  to  the  duel  of  Chapter  V,  the  system  performance 
can  be  characterised  by  the  expected  losses  per  raid  and  the  proba- 
blllty  of  killing  the  target  per  raid.  These  quantities  will  be  repre¬ 
sented  by  Ij^aja')  and  KR(a;a'),  respectively. 

The  optima’,  a- id  size  based  on  the  nominal  input  values  is  given 
by  che  following  a-ivtion  of  equation  (V*T-7) 


R  (a')  - 


ln(l-Kft) _ 

in  [l-sA(a')  Kpta' ;a' )" 


where  KpCaia’)  is  the  probability  of  kill  per  duel  as  a  function  of  a, 
for  given  a'.  The  probability  of  killing  the  target  per  raid  as  a  func¬ 
tion  of  a  for  given  a’  is  given  by 

KR(a;a' )  -  1  -  [l  -  sA(a)  %(a;a*  )]R(a' }  (14) 

and  the  expected  losses  per  raid  as  a  function  of  a  for  given  a'  is 
given  by 


LR(a;a’)  -  [l  -  SAz(a)  s0(a;a’)]  R(a’)  (15) 

where  S^aid')  is  the  prol.abiltty  of  the  aircraft  surviving  the  duel 
as  a  function  of  a  for  given  a'. 

An  adaption  of  tha  functional  equations  for  the  Pjj  duel  will 
serve  to  evaluate  ^(oua1).  From  equation  (V-24),  if  n  -  l  and  for 
1  <  1  <  3, 

?n(xi,  sr,  a;a’)  •  a)  sT(k{,  a)  rR(xl,  kj, ,  a)  (16) 
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and  from  equation  (V-26)  if  2  <  n  <  N  and  for  all  1  <  i  <  3, 

2 

fnl(xn*  sn*  a;a'^  "  ST(kn’  Su'kA»  a ^  2  pij<kn  •  a) 

j-1 

'  fn-l,j^xn’  sn-l»  a;a'^  ♦  ST(kn’  a}  p13(kn’  a) 

•  CrK(dA»  kA*  a>  ♦  VkA»  a)  <*  -  rK(dA*  kA*  a)) 

•  ^n-l,j<*n  -  dn»  sn-l»  <**«’>]  <17> 

Finally, 

Kjj(a;a')  «'f'Nl(xN,  sN,  a;a')  (18) 

The  survival  probability  SD(a;a')  can  be  evaluated  by  using  equations 
(10),  (11),  and  (12).  Equation  (IV-34)  relates  sn_j  to  sn. 

Numerical  Example:  Tactics  Selection 
Using  the  Eh  Duel  Raid  Model 

The  implications  of  user  uncertainty  with  respect  to  the  Ejj 
duel  raid  model  will  now  be  illustrated  by  using  the  example  problem 
that  was  introduced  In  Chapter  IV  and  further  discussed  in  Chapter  VII. 
The  nominal  Input  values  relating  to  acquisition,  aircraft  survival, 
and  weapon  effectiveness  are  unchanged  from  the  previous  example,  but 
a  range  of  uncertainty  will  now  be  associated  with  each  of  the  input 
quantities.  The  optimistic  and  pessimistic  value  of  each  parameter  is 
chosen  so  that  the  parameter  values  used  in  the  example  from  Chapters 
IV  and  VII  correspond  to  a*  -  0.5. 


163 


«**»*«**■’  m*m*a**' 1 


164 


o 

< 

CL 

2 

O 

h 

to 

o 

o 

< 

< 


lL 

O 

> 

-J 

m 

<5 

CO 

o 

o: 

CL 


o 

< 


00  0- 
o  >- 

0C 

w 

CO  > 

6  j 


L'J 

o 


2 

o 

to 

3 

o 

o 

■a 

-J 

< 


2 

lL 

o 

> 

►- 

-J 

CD 

< 

CO 

o 

cc 

CL 


< 

CL 

2 

O 

10 

o 

o 

<1 


2 

LU 

to 

O 

OJ 

to 

CO 

r> 

to 

Ll 

o 

>- 


rr> 

CD 

O 

cc 

CL 


Cl 

o  <f 

CL 

> 

.  cr 

.  u 
o  > 


Li 

c 

03 

4-> 

t- 

<0 

u 


V) 

o 


« 

u 


to 

.c 


(0 

6 


IT) 

6 

v 

6 

r<> 

d 

OJ 

o 


cn 

n 

o 

o 

< 


2 

UJ 

D 

O 

LJ 

CO 

CD 

r> 

CO 

u_ 

o 


>- 


-J 

CO 

< 

CD 

O 

or 

a 


a 


i 

i 

«n 

CM 


4 


to 

•H 

fc. 


f 


L 


& 


1 


i 


SINGLE  PASS  SURVIVAL  PROBABILITIES 
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12  3  4 

MODE  OF  ATTACK 


.930  .9e5  .990  .995  1.0 

PROBABILITY  OF  SURVIVING  THE  AREA  DEFENSE 
(ONE  WAY)  (SA) 


Fig.  26. ••Example  survival  probabilities  with  their 
ranges  of  uncertainty. 
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not  so  clear.  Since  the  goal  of  the  raid  is  to  make  ten  hits,  then 
the  cost  of  adopting  a'  •  0.0  may  not  be  Justified  since  that  philoso¬ 
phy  gives  ten  or  more  expected  hits  for  all  values  of  a.  It  might  be 
appropriate  to  plot  profiles  for  some  other  values  of  a'  to  further 
illuminate  the  decision. 

It  is  Interesting  In  general  and  might  be  of  particular  interest 
to  the  operational  planner  to  examine  the  attack  policies  that  are 
represented  by  the  three  values  of  a'.  Figures  29,  30,  and  31  are 
diagrams  of  the  attack  policies  for  a'  ■  0.5,  a'  »  0.0,  and  a’  •  1.0, 
respectively.  In  this  problem  the  operational  planner  would  be 
particularly  Interested  in  comparing  Figures  29  and  30.  Note  that  for 
a’  -0.5  (Figure  29)  the  minimum  number  of  passes  is  three,  the  maxi¬ 
mum  number  of  passes  is  four,  and  ■  3.  For  a'  ■  0.0  (Figure  30) 
the  maximum  number  of  passes  is  four  and  d£  ■  U.  Thus,  a’  -  0.0  leads 
to  a  more  conservative  policy  than  does  a'  •  0.5  as  would  be  expected. 
When  a'  “  1.0,  on  the  other  hand,  the  minimum  number  of  passes  is  five, 
thus  a'  «  1.0  is  the  least  conservative  policy  as  would  be  expected. 
Also,  a*  *  1.0  gives  k*  ■  1  for  all  cases  since  this  is  the  most  effec¬ 
tive  mode  and  there  is  assumed  to  be  no  associated  attrition  penalty. 

By  reflecting  on  these  results,  we  can  perhaps  get  some  Idea  of 
why  they  occur.  Taking  a’  ■  0.5  as  a  base  case,  lowering  a*  to  0,0 
results  in  higher  d*  because  of  the  decreased  survival  probabi 1  it ies 
at  a'  ■  0,0.  If  the  probability  of  survival  is  less,  anv  weapons  not 
delivered  on  a  given  pass  are  less  likely  to  he  delivered,  thus  the 
tendency  is  to  deliver  the  weapons  earlier  in  the  duel.  Still  taking 
a*  “  0.5  as  a  baso  case,  the  values  of  d£  are  generally  lower  when 
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rig.  29 . --Optimal  attack  policy  for  the  Eu  dual  raid  modal 


Fig.  31. --Optimum  attack  policy  for  the  duel  raid  model 
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a*  -  1.0  as  shown  in  Figure  31.  This  is  because  the  survival  proba¬ 
bilities  are  all  equal  to  1.0  when  a'  -  1.0  which  means  that  weapons 
not  delivered  on  the  current  pass  have  just  as  much  chance  of  being 
delivered  on  a  future  pass.  There  is  no  discounting  since  (S^Sy)  ■ 
1.0.  In  all  states  of  Figure  31,  the  remaining  weapons  are  allocated 
so  they  are,  as  nearly  as  possible,  evenly  distributed  among  the  re¬ 
maining  passes.  Note  in  Figure  31  that  ■  1  for  all  n.  This  Is 
because  mode  1  gives  the  highest  salvo  effectiveness  and  there  is  no 
attrition  penalty. 

numerical  Example;  Tactics  Selection 
Using  the  PR  Quel  Raid  Model 

To  illustrate  the  implications  of  user  uncertainty  with  respect 
to  the  duel  raid  model,  we  will  use  the  example  that  wa3  first  in¬ 
troduced  in  Chapter  V  and  was  further  discussed  in  Chapter  VII.  The 
acquisition  probabilities  are  given  in  Figure  25,  and  the  survival 
probabilities  are  given  in  Figure  26.  The  salvo  effectiveness  func¬ 
tions  are  similar  to  those  given  In  Figure  27  except  that  the  optimis¬ 
tic  and  pessimistic  limits  for  the  multiplier  4^  are  now  0.275  and 
0.225,  respectively.  The  values  of  6  are  unchanged.  The  salvo  effec¬ 
tiveness  function  is  Interpreted  here  as  the  probability  of  kill 
versus  salvo  size. 

Evaluation  of  equations  (16)  and  (15)  for  the  values  a*  -  0.0, 
a’  ■  0.5,  and  a'  ■  1.0  produces  the  profiles  of  system  performance 

that  are  shown  in  Figure  32,  These  curves  are  very  similar  to  those 
given  in  Figure  28  for  the  E^  duel  raid  model  and  the  same  sort  of 
comments  apply. 


Fig.  32. --Prof lies  of  system  performance 


The  system  performance  profiles  in  Figures  28  and  32  have  the 
feature  that  when  a  -  1.0,  Lg(a;a')  -  0.  This  is  because  all  survival 
probabilities  were  set  equal  to  1.0  at  their  optimistic  limit.  It  is 
Interesting  to  see  what  happens  when  this  is  not  the  case.  Suppose 
that  in  the  duel  example,  all  the  inputs  are  unchanged  except  that 
the  optimistic  limit  for  ST  and  Su  is  lowered  from  1.0  to  0.9975. 

This  results  in  the  modified  probability  of  survival  inputs  that  are 
shown  in  Figure  33.  The  resulting  system  performance  profiles  are 
shown  in  Figure  34. 

Comparing  Figure  34  with  Figure  28,  the  first  observation  is 
that  losses  no  larger  go  to  zero  when  a  -  1.0.  This  reflects  the  re¬ 
duced  optimistic  values  of  ST  and  Su.  The  values  taken  at  a  •  0,0  In 
Figures  34  and  28  are  essentially  the  same  but  the  values  taken  at 
a  »  1.0  have  changed  from  0.0  in  such  a  way  that  the  expected  loss 
curves  now  cross.  In  Figure  28,  the  tactic  fora'  *  1.0  is  clearly 
dominated  by  the  tactic  for  a'  -  0.5,  while  in  Figure  34,  there  is  a 
part  of  the  range  of  a  values  where  the  expected  losses  are  lower  when 
a’  •  1.0.  Note,  however,  that  for  all  values  of  a,  the  expected  hits 
are  much  lower  for  a'  «  1.0  than  for  a'  -  0.5  in  both  figures.  It  is 
not  clear  which  of  the  three  tactics  a  given  decision  maker  might 
choose  and  it  is  possible  that  he  would  like  to  see  results  for  more 
values  of  a'.  Nevertheless,  the  results  that  have  been  presented  have 
shed  some  light  on  the  implications  of  uncertainty. 
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Numerical  Example:  A  Designer's  Decision 
The  previous  examples  have  dealt  with  the  user's  decision, 
i.e.,  choice  of  the  best  tactic  for  a  given  system  in  a  given  situa- 
tion.  We  can  begin  to  consider  the  designer's  decision  by  way  of  an 
example.  Suppose  the  system  of  the  preceding  example  is  taken  as  a 
standard  system  which  is  to  be  modified  to  provide  improved  perfor¬ 
mance  in  that  particular  situation.  Suppose  the  following  alternative 
designs  are  available  where  A  represents  the  standard  system  while  B, 
C,  and  0  represent  equally  costly  modifications. 

A:  standard  system 

B:  provide  a  new  sensor:  set  P(A0)  -  P(A1)  -  1.0 
and  PIAqD)  -  PCA^D)  -  the  standard  system  values 
for  P(AjD)  (see  Figure  25) 

C:  increased  weapon  load:  Xjj  -  12  (xN  «  8  in  the 


N 


standard  system) 

0:  improved  first  pass  survival  probability: 

set  ST(kn)  -  Su(kn)  “  1.0  on  the  first  pass  for 
all  kn. 

When  comparing  alternatives,  we  should  in  principle  examine  all 
combinations  of  alternative  and  value  of  a'.  The  result  would  be  the 
most  desirable  combination  of  alternative  and  tactic.  Since  the  con¬ 
siderations  that  are  involved  are  largely  subjective,  it  is  difficult 
to  give  general  rules  that  would  lead  to  the  best  decision  in  a  particu¬ 
lar  situation.  Accordingly,  it  seems  reasonable  for  this  example,  and 
perhaps  as  a  starting  point  for  many  actual  evaluations,  to  compare 
alternatives  with  a’  set  at  a  nominal  value,  say  a'  •>  0,5. 
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Thus,  each  alternative  Is  evaluated  using  Its  own  best  tactic  for  the 
nominal  value  of  input  parameters. 

Figure  35  shows  the  profiles  of  system  performance  that  result 
for  the  various  alternatives  in  this  example  when  a'  °  0.5.  These 
profiles  display  the  effects  of  uncertainty  and  the  decision  maker's 
options  in  controlling  the  outcome.  We  cannot  say  how  a  decision  maker 
would  react  when  confronted  by  these  results,  but  we  can  point  out  some 
relevant  considerations. 

First,  it  is  interesting  to  study  the  seemingly  trivial  question 
of  whether  or  not  each  alternative  actually  provides  an  improvement 
over  the  standard  system.  First,  observe  that  B,  C,  and  □  all  show 
lower  expected  losses  than  does  A  for  all  a,  Next,  note  that  C's 
profile  of  expected  hits  is  essentially  the  same  as  that  of  A  so  we 
might  conclude  that  C's  performance  is  clearly  better  than  A’s.  If 
small  variation  of  expected  hits  is  a  goal,  then  B's  expected  hits 
profile  shows  less  variation  than  that  of  A,  therefore,  B  would  be  pre¬ 
ferred  co  A.  On  the  other  hand,  I)  shows  more  variation  of  expected 
hits  than  does  A  and  it  is  conceivable  that  D  would  not  be  preferred 
to  A.  This  is  a  very  interesting  possibility  since  if  uncertainty  is 
entirely  suppressed  and  we  compare  alternatives  on  the  basis  that 
a  ■  a'  “  0,5,  D  would  clearly  be  preferred  over  A,  B,  and  C.  This 
suggests  that  if  variation  of  expected  hits  is  of  primary  Importance, 
then  the  tactics  optimization  has  been  based  on  an  irconslstent  cri¬ 
terion.  If  so,  then  the  problem  must  be  reformulated. 

Assume  that  variation  of  expected  hits  is  not  so  important  as 
to  force  a  reformulation  of  the  problem,  i.e,,  alternative  D  is 


CTED  LOSSES 


lao 

preferred  to  alternative  A.  Even  with  this  assumption,  variation  of 
expected  hits  may  be  Important  enough  to  Influence  the  choice  among 
alternatives  B,  C,  and  D.  The  choice  in  this  example  might  in  fact 
be  viewed  as  a  tradeoff  between  minimizing  expected  losses  and  mini¬ 
mizing  the  variation  of  expected  hits.  To  minimize  expected  losses, 
alternative  D  would  be  selected;  alternative  B  minimizes  the  varia¬ 
tion  of  expected  hits.  In  this  particular  case,  alternative  C  might 
represent  a  reasonable  compromise. 

The  main  point  of  the  foregoing  discussion  is  that  if  uncer¬ 
tainty  had  been  completely  suppressed,  alternative  D  would  have  been 
chosen  without  hesitation.  When  uncertainty  was  considered,  a  new 
realm  of  considerations  was  revealed.  Alternative  D  may  no  longer  be 
selected.  It  may  even  be  decided  to  reformulate  the  problem.  This  mav 
or  may  not  be  reason  to  want  to  quantitatively  display  the  effects  of 
uncertainty,  depending  on  one's  point  of  view.  It  does,  however, 
illustrate  the  importance  of  uncertainty  and  it  shows  how  the  effects 
of  uncertainty  can  be  displayed. 

Another  Interesting  point  can  be  made  by  qualitatively  compar¬ 
ing  Figure  35  with  Figure  34.  Alternative  A  of  Figure  35  corresponds 
to  a'  -  0.5  in  Figure  34.  As  a  general  observation  from  comparing 
these  figures,  the  choice  of  a'  seems  to  be  of  comparable  Importance 
with  the  choice  of  system  design.  Among  the  designs  and  values  of  a' 
that  were  considered,  the  variation  in  outcomes  caused  by  changing 
the  design  with  fixed  a'  as  in  Figure  35  seems  to  be  no  greater  than 
the  variation  of  outcomes  that  is  caused  by  changing  the  value  of  a* 
while  keeping  the  same  design  as  in  Figure  34. 
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The  importance  of  tactics  can  be  further  emphasized  by  the 
following  extension  of  the  example.  Much  current  policy  for  air-to- 
ground  attack  establishes  a  limit  of  one  pass  per  sortie.  This  policy 
would  be  justified  if  area  defenses  were  negligible  and  first  pass 
attrition  was  small,  i.e.,  the  desired  level  of  effectiveness  could  be 
achieved  by  simply  increasing  the  raid  size  and  the  losses  would  be 
small  If  only  one  pass  is  made  per  aircraft.  Let  us  see  what  would 
happen  If  a  limit  cf  cne  pass  per  sortie  (N-l)  is  Imposed  in  the 
present  example  where  area  attrition  is  not  negligible  and  the  enemy 
has  sufficient  warning  so  that  first  pass  attrition  is  the  same  as  the 
attrition  or  later  passes.  The  resulting  profiles  of  system  pe i t Oi — 
mance  are  shown  in  Figure  36  for  alternatives  A,  B,  and  C.  For  the 
standard  system,  alternative  A,  the  expected  losses  have  roughly 
doubled  and  the  variation  of  expected  hits  has  greatly  Increased. 

Suppose  as  a  further  extension  of  the  example,  system  design 
alternatives  B  and  C  are  to  be  compared  under  the  restriction  of  one 
pass  per  sortie.  The  curves  for  B  and  C  in  Figure  36  are  applicable 
for  this  comparison.  On  this  basis,  B  shows  considerably  lower  losses 
and  less  variation  of  expected  hits  than  does  C.  In  Figure  35  where 
N  -  5,  the  choice  between  B  and  C  is  not  so  clear. 


CHAPTER  IX 


SUMMARY  REMARKS 

What:  Has  Been  Accompli  shed  ? 

According  to  the  general  problem  statement  given  at  the  be¬ 
ginning  of  Chapter  I,  we  set  out  to  "provide  a  systematic  method  for 
making  the  best  use  of  available  information"  relating  to  "a  rational 
selection  of  tactics  for  air-to-ground  attack  when  faced  by  uncer¬ 
tainty."  The  result  was  to  be  a  "quantitative  theoretical  structure." 

The  effort  to  achieve  these  goals  led  first  to  the  discussion 
of  basic  recursive  analysis  techniques.  Some  general  recursive  rela¬ 
tionships  were  developed  In  Chapter  II. 

It  was  then  shown  in  subsequent  chapters  how  these  recursive 
relationships  can  be  applteu  to  solve  progressively  more  complex 
duels.  Discussion  of  these  duels  started  in  Chapter  III  with  the  sim¬ 
plest  case  where  we  sought  the  allocation  of  weapons  among  passes  so 
as  to  maximize  the  expected  value  of  the  number  of  hits.  Probabilis¬ 
tic  target  acquisition  and  multiple  modes  of  attack  were  added  In 
Chapter  IV.  In  Chapter  V  the  recursive  relationships  were  applied  to 
a  duel  where  the  objective  was  to  maximize  the  probability  of  at  least 
one  hit  subject  to  suitable  constraints.  In  Chapter  VI  we  discussed 
the  more  general  case  where  an  aspiration  level  of  C  hits  was  estab¬ 
lished  and  finally  the  most  general  duel  was  solved  in  which  the 
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damage  level  or  utility  achieved  is  an  arbitrary  function  of  the 
number  of  hits. 

It  was  observed  that  In  all  of  these  duels,  simply  maximizing 
the  return  Is  not  sufficient.  The  cost  must  bo  considered.  Accord¬ 
ingly,  the  maximized  return  for  a  duel  was  expressed  as  a  function  of 
the  constraining  probability  of  aircraft  survival.  For  convenience, 
wo  generally  expressed  this  relationship  In  terms  of  attrition  rate 
rather  than  survival  probability  and  the  result  was  a  "return-versus- 
attrition  function"  that  characterizes  the  duel.  To  decide  which 
point  on  the  return-versus-attri tion  function  is  the  best  operating 
point,  multiple  aircraft  raid  models  were  developed  in  Chapter  VII  for 
both  single  and  multiple  target  raids. 

Finally,  in  Chapter  VIII  the  effects  of  user  uncertainty  were 
considered  and  a  method  was  discussed  for  presenting  to  the  decision 
maker  a  display  of  highly  digested  Information  as  to  the  effects  of 
uncertainty  and  his  options  to  control  the  outcome. 

It  is  a  matter  of  qualitative  judgment  as  to  whether  the  prob¬ 
lem  objectives  have  been  achieved,  but  it  certainly  seems  that  progress 
has  been  made.  A  theoretical  structure  has  been  developed  which  pro¬ 
vides  a  systematic  way  of  using  available  Information.  Uncertainty 
has  been  treated  and  practical  optimizing  methods  have  been  developed. 

Throughout  this  discussion,  the  aim  has  been  to  develop  methods 
that  can  actually  be  used  to  analyze  real  problems,  '.-,'e  have  attempted 
not  to  lose  sight  of  practical  application  for  the  sake  of  mathematical 
convenience.  There  are  essentially  no  results  that  depend  on  a  pecu¬ 
liarity  of  a  functional  form.  For  the  most  part,  the  methods  used  will 
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accommodate  arbitrary  functional  tonns,  e.g.,  salve  of  tort  (venoss  can 
take  any  value  within  the  limits  of  its  definition.  The  method  for 
dealing  with  user  uncertalntv  Involves  only  elementary  mathematics, 
but  It  seems  to  be  practical.  It  Rives  useful  recalls  without  making 
burdensome  demands  tor  input  data,  i.o.,  no  knowledge  of  probabllltv 
theory  is  required  to  bound  the  values  of  Input  variables.  Consider¬ 
able  thought  has  been  devoted  to  the  selection  of  criteria.  The 
emphasis  has  been  on  finding  adequate  solution  methods  based  on  real¬ 
istic  criteria.  Thi -  emphasis  has  led  to  the  consideration  of  a 
variety  of  duels,  multip’o  aircraft  raids,  multiplt  target  raids,  and 
to  user  uncertainty,  whereas  we  might  have  dwelt  on  finding  increas¬ 
ingly  elegant  solution  methods  for  some  problem  such  as  maximizing  the 
expected  value  of  the  number  of  hits  in  a  particular  aircraft  versus 
target  duel. 

In  discussing  the  motivation  for  this  study,  we  identified  a 
"user's  ueclslon  problem"  and  a  "designer's  decision  problem."  The 
results  that  have  been  obtained  seem  to  show  considerable  promise  for 
studying  users’  decisions.  These  results  should  also  be  useful  In 
studying  the  designer's  decision  but  we  are  left  with  the  dilemma 
that  is  discussed  in  the  following  section. 

A  Perplexing  Problem 

A  major  unsolved  problem  relates  to  the  effect  of  user  uncer¬ 
tainty  on  design  selection.  For  a  given  situation  which  involves  user 
uncertainty  the  methods  that  have  been  presented  in  this  report  can  be 
employed.  They  serve  to  determine  and  portray  to  the  decision  maker 
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Che  effects  of  uncertainty  and  his  options  to  control  svstem  per¬ 
formance  in  that  situation.  Both  tactics  selection  and  desist  selec¬ 
tion  can  be  studied  by  using  these  methods. 

Unfortunately,  in  comparing  system  designs,  we  must  generally 
consider  not  just  one  situation,  but  an  array  of  situations.  In 
accordance  with  die  Introductory  discussion,  the  application  of  de¬ 
cision  theory  requires  nunerical  values  that  represent  the  utility  of 
each  design  in  each  of  the  situations.  The  best  we  have  been  able  to 
do  Is  to  quantitatively  display  the  effects  of  uncertainty.  This  pro¬ 
vides  a  basis  for  the  decision  maker  to  choose  among  alternatives  in 
a  given  situation  but  it  does  not  provide  the  desired  utility  measure. 

How  then  Is  the  decision  maker  to  make  his  choice  when  many 
situations  are  Involved?  One  course  of  action  is  to  limit  the  number 
of  blocks  in  the  decision  matrix  and  present  a  complete  analysis  in¬ 
cluding  the  effects  of  user  uncertainty  for  each  block.  If  the  de¬ 
cision  matrix  is  sufficiently  small,  perhaps  the  decislcn  maker  can 
comprehend  the  meaning  of  such  results  and  make  a  rational  decision 
accordingly.  This  approach  may  at  lease  prevent  him  from  making  a 
completely  irrational  decision. 

An  alternative  is  to  entirely  suppress  the  user's  uncertainty 
when  making  design  comparisons  (this  is  what  is  usually  done).  The 
techniques  discussed  in  this  report  allow  us  to  optimize  tactics  in 
each  block  if  user's  uncertainty  is  ignored.  This  accomplishment  is 
well  worthwhile.  It  reduces  our  vulnerability  to  the  possibility  that 
tactics  are  at  least  as  important  as  system  design.  Non  optimal  tac¬ 
tics  can  be  useless  or  even  misleading  as  a  basis  for  comparing  desig.s. 
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APPENDIX  A 

DEVELOPING  RECURSIVE  REUT IONSHIPS  FROM 
A  NON  RECURSIVE  PROBLEM  STATEMENT 


In  this  appendix  we  will  develop  a  non  recursive  statement  of 
the  moat  general  problem  that  Is  treated  In  Chapter  IV,  l.e. ,  the  E^ 
duel.  The  recursive  relationships,  equations  1V-27  and  IV-28,  which 
are  the  principal  results  of  Chapter  IV,  will  then  be  developed  from 
the  non  recursive  statement  of  the  problem  by  using  a  method  similar 
to  that  of  Neuhauser  (22).  It  will  also  be  Indicated  how  the  approach 
that  is  followed  in  this  appendix  can  be  applied  to  the  most  general 
problem  that  Is  treated  in  Chapter  V,  l.e.,  the  PK  duel. 

In  the  general  E^  duel,  we  seek  to  maximise  the  expected  value 
of  the  number  of  hits  achieved  In  a  duel  that  Includes  probabilistic 
target  acquisition  and  multiple  modes  of  attack  subject  to  constraints 
on  the  number  of  passes,  N,  the  number  of  weapons  available,  x^,  and 
the  probability  of  the  aircraft  surviving  the  duel,  s^. 

The  notation  used  In  this  appendix  Is  the  same  as  that  used  in 
the  main  body  of  this  work  with  the  following  modifications.  Let  Jn 
denote  the  Markov  state  of  the  system  at  stage  n  -  1.  Thus  the  vector 
(jN,  Jjj-i,  j})  denotes  one  of  the  possible  sequences  of  Markov 

state  transitions  and  the  probability  of  that  sequence  occurring  is 
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where  p,  .  (k  )  Is  the  probability  that  the  system  will  be  in  state 
•>nelJn  n 

jn  at  stage  n-1  given  that  the  system  is  in  state  J  j  st  stage  n, 

Thi9  transition  probability  depends  on  kR,  the  node  of  attack  at 
stage  n. 

Let  Pjj  denote  a  policy  for  a  duel  in  which  the  number  of  passes 
is  not  to  exceed  N.  The  policy  is  a  set  of  Instructions  that  indicates 
the  course  of  action  for  the  pilot  to  follow  at  every  stage  for  every 
state  that  the  system  might  be  in,  i.e.,  the  policy  tells  the  pilot 
what  mode  of  attack  to  use  and  Che  number  of  weapons  he  should  seek  to 
deliver  on  the  next  pass  as  a  function  of  the  number  of  passes  remain¬ 
ing,  the  current  Markov  state,  the  number  of  weapons  remaining,  and  the 

current  constraining  attrition.  We  will  denote  bv  P*.  that  part  of  the 

N 

policy  that  specifies  (xM,  sN)  and  dM.  (xN.  s„)  at  the  first 

NJNel  JNel  "  W 

pass.  Let  denote  that  part  of  the  policy  that  specifies 

knj  j  (xn’  V  and  dnJn>i  <Xn’  for  a11  COBbln*tio"s  of  n*  Jn*i» 
xR,  and  sn  such  that  l<n<N  -  1,  l  <  <  I,  xn  g  Sx  ,  and  sn<r  sg 

1  7 

(See  Footnotes  *  )  Let  Sp,  denote  the  set  of  all  possible  policies 

N 

at  pass  N  and  let  Sp^  denote  the  sec  of  all  possible  policies  for  the 
remaining  N-1  passes. 


*Note  that  since  some  of  the  transition  probabilities  are  zero 
(see  Figure  6),  some  of  the  above  states  may  not  be  reachable,  so  it  is 
theoretically  not  necessary  for  the  policy  to  cover  all  of  them.  It 
la,  however,  quite  complex  to  determine  which  of  the  above  states  can 
be  reached  and  which  ones  cannot  and  for  our  purposes,  this  considera¬ 
tion  will  be  ignored. 

2 

As  in  the  main  body  of  this  work,  the  functions  knj  (x^,  dn) 

and  d  |  (x_,d  )  are  abbreviated  k_  and  d  when  they 

n  j  ^  j  11  11  n  n 

appear  In  the  argument  of  other  functions. 


190 


(  N 

fNl(X«)  -  Max  4  2  -  -  -  2  n  P1  ,1  (kn) 

PNesP‘  ^  over  over  n-1  " 


H  Jn 
PN-leVl 


-  -  2  n 

over 

•*1 


[  ST(kN)  fj^(dN,kN)  ♦  S^Oe^)  Su(kK)  ST(kN_j,)  rjN_1^dN-l,kN-l) 


*  '  “  -  ♦  ST(kN-JJel-  n  sT<kn*  Su(kn)  rjM  (dN-J»l,kN-/*-l) 

n-N-JU2  N~I+l 


♦  S>p(kj)  tt  S»j(k 

n~2 


n>  su<kn>  j| 


(3) 


where  1  -  and  XN  -  (x^S-j).  The  variable  i  has  Che  same  meaning 

here  with  respect  to  stage  N  that  it  did  in  the  main  body  of  this  work. 
The  variable  1  indicates  the  Markov  state  of  the  system  at  stage  N. 

This  expression  can  be  made  more  compact  by  using  the  index  X 

to  give 


Ni 


<V 


Max 

PN  €  SP 


.{ 


PN- 1  ^  SP, 


2  -  -  -  £ 

over  over 

ji 


r n  Pi  4  o<n)l 

[  n-l  JnelJn  n  J 


N  r 

£  [  ST(kN-iei: 


N-l 


N 

n 

n-N-Jle2 


ST<kn>  Vkn>  rj 


N-4+1 


(4) 

*dN-j?^l»kN-JelJ 


where  for  an  arbitrary  function  g(n)  we  define 


N 

tt  g(n)  =  1 
n«Nel 


(5) 


1 
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The  above  maximization  must  be  carried  out  subject  to  tvo  con¬ 
straints.  First,  the  nusber  of  weapons  available  for  n-1  passes,  x  , , 

n-  l 

is  the  number  of  weapons  available  for  n  passes,  xr,  less  the  number 
that  is  delivered  on  pass  n.  Recalling  that  weapon  delivery  only 
occurs  at  pass  n  when  Jn  -  3,  we  have  the  following  relationship  for 
all  1  <  n  <  N. 


Vl  -  xn  ;  Jn  -  l*  2 


*  *n  *  dn  *  ^  -  3 


(6) 


We  also  require  that  the  probability  of  the  aircraft  surviving  the  duel 
be  at  least  s^  or  symbolically 


N 


17  ST(kn}  Su<kn>  -  SN 
n-1 


(7) 


Thus,  in  Che  moot  general  problem  of  Chapter  IV,  we  seek  to  per¬ 
form  the  maximization  Indicated  by  equation  (A)  subject  to  the  con¬ 
straints  indicated  by  equations  (6)  and  (7). 

Isolating  the  term  corresponding  to  Ji  -  1  in  equation  (A)  gives 

r  N 

fN1(XN)  -  flax  "S  2---  £  Tip.  (k  ) 

Pi  €  Sp,  L  over  over  n-1  JnelJn 


over 


over 

ii 


rN-l 


N-1 
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Considering  the  Markov  state  transitions,  note  that  the  first  tens  of 
•  3  *n  equation  (3)  depends  only  on  JN  and  the  second  term  depends 
only  on  »jj)-  Thus,  we  can  write 


f 


Ni 


0U.)  "  M  3x  T  ^  P«  i  r  ^kN ^  ri 

Pj,  €  Sp,  L  over  JNelJN  "  L  T  N  JN  N  N 


N  Jn 

PN-1  6  SPNM 


♦  2 
over 

Jn-i 


N-l 

2  n  Pj  ,1  <kn> 

over  n-l  3r»Vn 


(9) 


H 


N  N  *1 

X  -2  Sx<kN-^l)  n.N?^2  ST<kn>  Su(kn>  rJH~^l<dN’^1,kN“^l)]  J 


Now  by  factoring  the  quantity  ST<kN)  Su(kN>  out  of  the  second  term  in 
equation  (9)  and  by  replacing  the  Jl  by  A  *1,  equation  (9)  becomes 


fNi(XN)  -  Max 

PN  €  Sp 


PN-l  €  SP 


■f  2  PJn*iJn°C**)  [ST(kN)  rjN(dN»kN) 
f  v.  over  1  w 


Jn 


ST(kN) 


PN»  1 

N-l 

z 

-  -  L  n 

over 

over  n-l 

Jn-i 

Jl 

N-l 


w  s«<k">  ao> 

Since  S^Ck^)  Su(kN)  >  0,  the  outside  quantity  {•}  In  equation  (10) 
Is  a  monotonlcally  nondecreasing  function  of  inside  quantity-^  ”  , 
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stralnts. 
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By  raplacing  N  by  N-l  In  equation  (4),  we  recognise  that  the 

that  appears  In  equation  <11)  Is  the 


quantity  Max 

PN-1  ^  SP 


{  } 

N-l 


sane 


as  j^<XN_i>  so  equation  (11)  can  be  written  as: 

fNlCXN)  -  ^  Max  £  '’jNeliN0'  )  [  ST<kN)  r]N(dN*kN) 

N  PN  N 


ST(kN)  Su(kN)  fjj.it  j^(X{j- 1^ 


(14) 


where  ■  ^xn-i»®n-l^* 

To  see  that  equation  (14)  is  the  same  as  equation  (1V-28)  for 

n-N,  we  use  the  fact  that  fi(dN,kN)  -  r2(dN,kN)  "  °»  wn,le  r3(dN»kN^  " 

rH(dN,kN).  Substituting  these  into  equation  (14)  and  using  equations 

(12)  and  (13)  and  noting  the  definitions  of  P'  and  Sp.  gives 

"  rN 

fNl<XN*®N)  “  4  2  PiJn^N5  ST*kN*  Su(kN)fN-l,jixN» 

*n  *  skN<8N>  l  3n-1  N  N 

dN  €  sdN<xN> 


ST(kN)Su(kN)  *  *  Pl3(kn)  [  ST^kN^  rHWN»kN) 


ST(kN>  Su<kN>  fN-l,3<xN  ’  V  sT(kN)!u(W(|)>  ^  } 


(15) 


The  trivial  generalisation  of  replacing  N  by  n  shows  chat  equation  (15) 
holds  for  all  2  <  n  <  N.  Furthermore,  If  for  all  1  <  Jj  <  3, 

‘oj^o*  = 


0 


(16) 
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Then  equation  (15)  becomes 

fli(xl»sl>  “  p13<kl^  sT(ki)  rH<dl»kl*  j"  07) 

kl€Ski(Sl)l  J 

di€Sdi(x,,) 

which  is  the  same  as  equation  (IV-27). 

Now  consider  how  the  approach  that  has  been  followed  In  this 
appendix  can  be  applied  to  the  aost  general  problem  that  la  treated  in 
Chapter  V,  l.e. ,  the  PK  duel-  The  major  change  from  the  previous  dis¬ 
cussion  In  this  appendix  Is  that  we  must  replace  expression  (2)  with 
the  following  expression  (18)  which  represents  the  probability  of  at 
least  one  hit  for  a  given  sequence  of  Markov  state  transitions. 

On*  — «Ji>  *nd  for  a  given  policy.  Note  that  for  the  PR  duel,  if 
jN  •  3,  then  «*3(dn,kn)  -  rK(dn,kn).  Th,s  expression  can  be  rationalised 
by  using  the  same  type  of  reasoning  that  led  to  equations  (5)  through 
(8)  of  Chapter  V. 

ST(kN)rjN(dN,kN)»ST(kN)Su(kN)ST(kNi>l)  [l-rj^(dN,kN)]  rJN-1  ^dN«  I  »*N-1' 

♦ST(kN)Su(kN)ST<kN_i)Su^kN-l'ST^kN-2^  C1*rJN*dN»kN^ 

Cl“rjN_l^dN-l»kN-l^  rJN-2*dN-2*kN-2> 

♦ 

N 

♦  ST(kN.j^4i)  tt  SfOtnJSuCkn)  £l-rj  (dn,kn)^  rjN.|*i  ^dN-4*l*kN--lel) 

n»N-J!+2  n  *’”-**  i 

♦ 

a 

I  N 

♦  ST(k1)  IT  ST(kn)Su(kn)  [1-rj  (dn,kn)]  r,  <d1,kl)  (18) 

n»2  n  ■>1 


1V6 


The  details  of  the  development  will  not  be  given  but  the  argu¬ 
ment  Is  identical  to  that  given  for  the  E^  duel.  The  changes  that  must 
be  made  to  apply  the  argument  of  this  appendix  to  the  P^  duel  follow  in 
a  straightforward  manner  from  the  use  of  expression  (18)  in  place  of 
expression  (2). 


APPENDIX  B 


SCME  PROOFS  RELATED  TO  THi:  MATERIAL 


IN  CHATTER  II 


This  appendix  contains  a  proof  of  the  comment  that  immediately 
follows  the  statement  of  the  equivalence  condition  for  the  deterministic 
decision  process  in  Chapter  II,  page  31.  This  appendix  also  contains 
a  proof  of  the  comment  tha'  Immediately  follows  the  statement  of  the 
monotonlclty  and  equivalence  conditions  for  the  Markovian  decision  pro¬ 
cess  in  Chapter  II,  page  3b. 

First,  relating  to  the  deterministic  decision  process,  we  will 
prove  that  "it  follows  from  the  definitions  of  monotonlclty  and 
fn_|(Xn-l)  thl*t  th®  function  g  [Xj,,  Dn,  fn.|(Xn_1)]  represents  the 
maximum  return  that  is  obtainable  from  the  n  stage  system  for  given  X„ 
and  Dn." 

Let  Pn  denote  a  sequence  of  decisions,  (Dn,---  D|),  or  "policy" 

for  stages  n,-  —  ,1.  Let  Sp  (Xn)  denote  the  set  of  all  feasible  policies 

n 

for  stages  n,---,l  where  Sp  (X^  depends  on  Xn.  Let  f^(Pn)  denote  the 
n  stage  return  that  is  realisable  by  ustng  policy  Pn  and  let  Fn*(Xri) 
be  the  optimum  n  stage  policy  as  a  function  of  Xj,  (to  be  abbreviated 
Pn*>  so  that  considering  the  definition  of  ^(X,,), 


fn-l<Pn-l>  <  fn-l<Pn-l*>  -  fn-l<*n-l> 


for  all  Pn-1  €  Sp^^XH.j). 
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Now,  considering  the  definition  of  monotoniclty,  it  follows 

that 

«ntV  Dn*  Cl<Pn-l>]  <  «ni>n»  Dn»  fn-l<Vl>]  <2> 

for  all  Pn.j  €  Sp^  ^(Xn-i)  and  for  every  X„  €  end  Dn  c  sDn(xn). 
This  completes  the  proof. 

Next,  relating  to  the  Markovian  decision  process,  we  will  prove 
that  given  monotonlclty  and  equivalence,  "the  function 
gnij  Lxn«  ®n*  fn-l, represents  the  maximum  expected  value  of 
return  that  is  obtainable  frcm  the  n  stage  system  for  given  values  of 
Dn.  1.  and  J.» 

Let  Pn  denote  a  policy  for  stages  n,-— ,1.  Since  the  sequence 

states  that  the  system  will  occupy  at  stages  n- 1,--- , 1  is  not  known, 

the  policy  must  completely  define  the  value  that  the  decision  vector 

is  to  take  as  a  function  of  the  state  of  the  system  at  all  stages, 

l.e.,  Pn  must  define  <Dn <X^) ,  Dn.j(X,  .j) ,  D1(X1))  for  all  feasible 

sequences  (Xj,,--- ,Xj  )  where  X„  is  the  state  vector  which  includes  the 

Markov  state.  Let  SP  (Xn)  denote  the  set  of  all  feasible  policies  for 

n 

stages  n,— -,1  where  Sp^Otj,)  depends  on  X„.  Let  f^  (Pn,  X^)  denote 
the  expectod  value  of  the  n  stage  return  that  is  realisable  if  the 
system  Is  In  state  X^  at  stage  n  and  policy  Pn  is  used.  Let  P^X,,) 
be  the  optimum  n  stage  policy  as  a  function  of  X,,  (to  be  abbreviated 
Frt*)»  so  considering  the  definition  of  fn.i(Xn»l)* 

fn-l^pn-l*  —  ^n-l^pn-l»  "  fn-l^*n-l^ 


for  all  Vi€Vr 
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Now  If  j  das lgn*tes  the  Marfcov  state  of  the  system  at  stage  n-1, 
the  foregoing  can  be  written 

*n-l,j<pn-l»  *n-l>  £  Cl,J<Cl.  *n  [)  -  fn.ifj<X„_i>  (3) 
for  all  Pn_i€Spn  i(Vl>.  Vl*  S^,  I  <  J  <  I. 

It  follows  from  the  above  and  from  the  definition  of  monotonietty  that 

*nij  IXt*  Dn*  Vl.j^n-l’Vl^  -  *nij  [*n»  Dn»  fn-l,J^n-i^3  <4> 

for  all  i,  j,  X^»  Dn,  Pn.j  in  their  respective  sets.  From  the  fore¬ 
going  and  the  equivalence  property,  it  follows  that 

gnij  D*n»  Dn»  Cn-l,J^*n-p3  ,s  th®  ®axlB,u"  expected  value  of  the 
n  stage  return  for  given  ,  Dn,  i,  J.  This  completes  the  proof. 
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APPENDIX  C 


ANOTHER  SOLUTION  METHOD  FOR  MARKOVIAN 
DECISION  PROCESSES 

It  may  have  occurred  to  the  reader  that  the  general  E^  duel, 
equations  (IV-27)  and  (IV-29)  and  the  general  ly  duel,  equations 
(V-24)  and  (V-26)  can  also  In  principle  be  solved  by  Howard's  value 
Iteration  method  (17).  It  Is  Interesting  to  see  what  Is  Involved  If 
tnat  method  Is  applied  directly  to  the  examples  that  have  been  dis¬ 
cussed  herein.  Howard's  notation  will  be  used  in  this  discussion. 

To  apply  value  Iteration  to  a  discounted  Markovian  decision 
problem,  the  following  recursive  relation  is  used  (17,  p.  80). 


where 


N 

vj  (n  ♦  1)  -  Max  [d*  ♦  g  £  Pm  v.  (n)] 
k  j-l 


N 

r  „  k  _  k 


(l) 


n 

1 

k 

fi 


pij 


stage  index 

Markov  state;  1  -  1 ,--- ,N 
Index  on  decision  alternatives 
discount  factor;  analogous  to  aircraft  survival 
probabi llty 

i  to  j  transition  probability  under  decision  k 
reward  associated  with  the  i  to  j  transition 


under  decision  k 
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To  solve  the  E^  duel  example  of  Chapter  IV  by  use  of  equation 
(1),  the  value  of  the  Markov  state  variable  1  must  completely  charac¬ 
terise  the  state  of  the  system.  In  the  example  of  Interest,  the 
state  of  the  system  is  characterised  by 

a.  xn  «  The  number  of  bombs  remaining:  8  levels. 

b.  sn  -  The  constraining  survival  probability:  12  levels. 

c.  The  acquisition  status:  2  levels,* 

Since  these  variable  values  can  occur  in  all  combinations,  the  number 
of  levels  required  for  the  Markov  state  variable  Is  N  •  8  x  12  x  2  ■ 
192.  Thus,  to  solve  the  example  problem  by  value  Iteration,  each  of 
the  transition  and  reward  matrices  has  192  x  192  -  36,864  elements 
that  must  somehow  be  evaluated  and  accounted  for  in  the  calculations. 

Solving  the  Chapter  V  example  of  the  duel  Involves  even 
larger  sise  matrices.  The  state  of  the  system  at  any  stage  Is  charac¬ 
terized  by  all  of  the  previous  factors  In  addition  to  which  the  status 
of  the  target  must  be  specified,  i.e.,  it  Is  either  dead  or  alive. 

This  Is  required  because  the  pilot  does  not  know  when  target  kill  Is 
achieved  and  therefore  the  duel  may  continue  after  the  target  has  been 
killed  but  with  a  different  reward  per  stage.  The  result  is  that  for 
the  Pjj  duel  example,  N  -  8  x  12  x2x2  -  384  levels  are  required  for 
the  Markov  state  variable.  The  transition  and  reward  matrices  each 
have  384  x  384  -  147,456  elements  to  somehow  account  for. 


*Note  that  for  this  formulation  the  acquisition  status  can  be 
characterized  by  two  levels,  i.e.,  the  target  has  either  been  acquired 
or  It  has  not  been  acquired.  In  the  example  of  Chapter  IV,  the  acqui¬ 
sition  status  required  three  characterizing  levels  because  of  the 
relationship  between  acquisition  status  and  weapon  delivery. 
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In  solving  the  preceding  examples  by  the  methods  presented 
herein,  the  Markov  state  variable  has  t?  .ee  levels  and  the  matrix  of 
transition  probabilities  has  nine  elements.  The  extent  of  the  compu¬ 
tation  seems  to  be  roughly  comparable  otherwise.  In  effect,  our 
method  is  equivalent  to  equation  (1).  The  two  methods  perform  the 
sane  operations  and  arrive  at  the  same  result,  but  the  former  is  con¬ 
siderably  more  efficient  and  easier  to  apply  to  the  problems  that  are 
of  Interest  here. 
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within  their  ranges  of  uncertainty. 
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